Mathematic Courses
===

There are many aspects of mathematic. However, as a machine learning / deep learning researcher, I collect several high-quality mathematical courses that benifit for machine learing.

>[!Important]
> 1. The courses I listed here may not contain corresponding video lectures.
> 2. Please mind that you should follow the copyright regulations.

## Calculus

Calculus is not really important in **deep learning**, as you just need to know how to conduct differntiation :)

>[!Caution] Claim
However, if you are considering conducting research related to **learning theory**, **statistical machine learning**, and other interesting topics, you need to know the detail of calculus, even a little knowledge of analysis.
I believe that if I were a Ph.D. candidate, it is *shameless* if I do not know the mathematical knowledge. I collect several excellent calculus course here.

### MATH4007 Calculus I-IV, by National Taiwan University

>[!Tip] Information
> 🧑‍🏫 **Instructor:** Ya-Ju Tsai
> 🏫 **Offering Institution:** National Taiwan University
> 📅 **Semester:** 2023 Fall + 2024 Spring; 2024 Fall + 2025 Spring
> 🌐 **Language:** Traditional Chinese
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Lecture Videos

:::info

**Lecture Videos** ⬇️

{%youtube aCcV8UFmYp8%}

Pleasea also see the YouTube Channel below ⬇️:

{%preview https://www.youtube.com/@yjtsaimath/courses %}
:::

### MATH4007 Calculus I-IV, by National Taiwan University

>[!Tip]
> 🧑‍🏫 **Instructor:** Kuo-Wing Tsai
> 🏫 **Offering Institution:** National Taiwan University
> 📅 **Semester:** 2022 Fall + 2023 Spring
> 🌐 **Language:** English
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

{%preview https://ocw.aca.ntu.edu.tw/courses/111S102 %}

{%preview https://ocw.aca.ntu.edu.tw/courses/111S103 %}

{%preview https://ocw.aca.ntu.edu.tw/courses/111S204 %}

{%preview https://ocw.aca.ntu.edu.tw/courses/111S205 %}

:::

## Mathematical Analytics

This is the fundamental part for math. I will only illustrate several high-reputation courses here.

### MATH142a Introduction to Analysis I, by UCSD

>[!Tip]
> 🧑‍🏫 **Instructor:** Yuriy Nemish
> 🏫 **Offering Institution:** University of California San Diego
> 📅 **Semester:** 2021 Winter
> 🌐 **Language:** English
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus


>[!Note]
>Please see the course website below. ⬇️
>{%preview https://mathweb.ucsd.edu/~ynemish/old/2021/142a/index.html %}

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

{%preview https://www.bilibili.com/video/BV1DF411J78S/?vd_source=66bf054dfd8af40f1c6224b27df8d39c %}
:::

## Linear Algebra

Linear Algebra is really important in many areas. I shall recommand some more *mathematical tasted* linear algebra courses here.

### MATH1103 / MATH1104 Linear Algebra, by NTU

>[!Tip]
> 🧑‍🏫 **Instructor:** 莊武諺
> 🏫 **Offering Institution:** National Taiwan University
> 📅 **Semester:** 2021 Fall, 2022 Spring
> 🌐 **Language:** Traditional Chinese
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

:::success
| 週次 | 日期 | 課程內容 |
| --- | --- | --- |
| 第1週 | 09/22, 09/24 | 09/22: fields, vector spaces, subspaces. <br>09/24: subspaces, linear dependence, basis. |
| 第2週 | 09/29, 10/01 | 09/29: linear dependence, basis. <br>10/01: basis, replacement theorem, dimension. |
| 第3週 | 10/06, 10/08 | 10/06: linear transformation, kernel, range, dimension theorem. <br>10/08: dimension theorem, projection. |
| 第4週 | 10/13, 10/15 | 10/13: matrix representations. <br>10/15: matrix representations, invertible linear transformations. |
| 第5週 | 10/20, 10/22 | 10/20: invertible linear transformations, change of coordinates. <br>10/22: elementary matrices. |
| 第6週 | 10/27, 10/29 | 10/27: elementary matrices, Gaussian elimination. <br>10/29: Gaussian elimination, determinant of order 2. |
| 第7週 | 11/03, 11/05 | 11/03: determinants. <br>11/05: determinants, Cramer's rule, adjoint matrices, determinants in terms of permutations. |
| 第8週 | 11/10, 11/12 | 11/10: midterm. <br>11/12: diagonalization. |
| 第9週 | 11/17, 11/19 | 11/17: diagonalization. <br>11/19: invariant subspaces, Cayley-Hamilton theorem. |
| 第10週 | 11/24, 11/26 | 11/24: 在新生103發還期中考考卷 <br>11/26: Jordan forms, generalized eigenspaces. |
| 第11週 | 12/01, 12/03 | 12/01: Jordan forms. <br>12/03: no class. 全校運動會 |
| 第12週 | 12/08, 12/10 | 12/08: Jordan forms. <br>12/10: Jordan forms. |
| 第13週 | 12/15, 12/17 | 12/15: exponential of matrices. <br>12/17: systems of first order differential equations, minimal polynomials. |
| 第14週 | 12/22, 12/24 | 12/22: rational canonical forms. <br>12/24: rational canonical forms. |
| 第15週 | 12/29, 12/31 | 12/29: rational canonical forms. <br>12/31: no class. |
| 第16週 | 01/05 | 01/05: final. |
:::

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

{%youtube Uph61S5pJfU %}

{%preview https://www.youtube.com/playlist?list=PLQqeHUV7RZ2PJFB54P2Wrf0LmRaIIGyET %}

{%preview https://www.youtube.com/playlist?list=PLQqeHUV7RZ2NZTtKnbQ6v-IlgOP260HNk %}
:::

### MATH1101 / MATH1102 Linear Algebra, by NYCU

>[!Tip]
> 🧑‍🏫 **Instructor:** 莊重
> 🏫 **Offering Institution:** National Yang Ming Chiao Tung University
> 📅 **Semester:** 2011 Fall, 2012 Spring
> 🌐 **Language:** Traditional Chinese
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

:::success
| 章節 | 內容 |
| --- | --- |
| 第一章 | Vector Space |
| 第二章 | Linear Transformation and Matrices |
| 第三章 | Elementary Matrix Operators and Systems of Linear Equations |
| 第四章 | Determinants |
| 第五章 | Diagonalization |
| 第六章 | Inner product space |
| 第七章 | Canonical Forms |

:::

#### Lecture Video

:::info
**Lecture Videos** ⬇️

{%preview https://www.youtube.com/playlist?list=PLj6E8qlqmkFtjxknKFtdxc1_SxNBXgpbo %}

{%preview https://www.youtube.com/playlist?list=PLj6E8qlqmkFsU6_lxu7soHGgI30IdpA7F %}

:::

## Probability and Statistics

Probability and Statistics are really useful and important regarding to learning theory.

### MATH280A: Probability Theory I, by UCSD

>[!Tip]
> 🧑‍🏫 **Instructor:** Todd Kemp
> 🏫 **Offering Institution:** University of California San Diego
> 📅 **Semester:** 2024 Fall
> 🌐 **Language:** English
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

>[!Note]
>Math 280A is the first quarter of a three-quarter graduate level sequence in the theory of probability. This sequence provides a rigorous treatment of probability theory, using measure theory, and is essential preparation for Mathematics PhD students planning to do research in probability. A strong background in undergraduate real analysis at the level of Math 140AB is essential for success in Math 280A. In particular, students should be comfortable with notions such as countable and uncountable sets, limsup and liminf, and open, closed, and compact sets, and should be proficient at writing rigorous epsilon-delta style proofs. Graduate students who do not have this preparation are encouraged instead to consider Math 285, a one-quarter course in stochastic processes. See also [this page](http://www.math.ucsd.edu/~williams/probcourse.html), maintained by Ruth Williams, for more information on graduate courses in probability at UCSD.
>According to the UC San Diego  [Course Catalog](http://ucsd.edu/catalog/courses/MATH.html), the topics covered in the full-year sequence 280ABC include the measure-theoretic foundations of probability theory, independence, the Law of Large Numbers, convergence in distribution, the Central Limit Theorem, conditional expectation, martingales, Markov processes, and Brownian motion. The topics for the current version of Math 280A cover everything up to the Law of Large Numbers (with additional topics).
**Prerequisite:**  Students should have mastered the fundamentals of real analysis in metric spaces, as covered in MATH 140AB, before taking this course. An undergraduate course in probability, comparable to MATH 180A, and further courses in stochastic processes, comparable to MATH 180BC, would also be an asset, but are not absolutely necessary.

>[!Caution]
>Please See the course website below.
{%preview https://mathweb.ucsd.edu/~tkemp/280A/ %}

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

{%preview https://www.youtube.com/channel/UCeKkMyeKBnec9Y3I_eI2BNQ %}

:::

## Optimisation

Optimisation, a special topic, useful in deep learning and other research areas.

There are two kinds of optimisation courses,

* Traditional Optimisation. e.g., Operations Research, Convex Optimisation
* Machine Learning / Deep Learning Area Optimisation.

### 21-292 Operations Research I, by CMU

>[!Tip]
> 🧑‍🏫 **Instructor:** David Offner
> 🏫 **Offering Institution:** Carnegie Mellon
> 📅 **Semester:** 2023 Spring, 2025 Spring
> 🌐 **Language:** English
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

>[!Note]
> **Students will**,
> * Formulate many types of problems as linear programs or integer programs.
> * Find optimal solutions for linear or integer programs using Gurobi optimizer software.
> * Perform the simplex algorithm for solving linear programs and justify why it works.
> * Describe the simplex algorithm, and optimal solutions to a linear program geometrically.
> * State and prove the strong and weak duality theorems for linear programming, and complementary slackness conditions for optimality.
> * Formulate primal-dual algorithms for analyzing optimization problems such as shortest path and min cost perfect matching.
> * Analyze the efficiency of optimization algorithms.
> * Analyze approximation algorithms for NP-hard problems such as min cost set cover.
> * Perform sensitivity analysis on an optimal solution to a linear program.
> * Use total unimodularity to establish the integrality of solutions to some linear programs.
> * Generalize optimality conditions from linear programming to nonlinear programs as KKT conditions.

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

2025 Spring Semester course website:

{%preview https://canvas.cmu.edu/courses/45012 %}

2023 Spring Semester course website:

{%preview https://canvas.cmu.edu/courses/33035 %}
:::

### IM2010 Operations Research, by NTU

>[!Tip]
> 🧑‍🏫 **Instructor:** Ling-Chieh Kung
> 🏫 **Offering Institution:** National Taiwan University
> 📅 **Semester:** 2025 Spring
> 🌐 **Language:** English
> 🎓 **Level:** Undergraduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

>[!Note]
>

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

{%preview https://cool.ntu.edu.tw/courses/47152/ %}

:::

### 10-725 Convex Optimization, by CMU

>[!Tip]
> 🧑‍🏫 **Instructor:** Yuanzhi Li
> 🏫 **Offering Institution:** Carnegie Mellon University
> 📅 **Semester:** 2023 Spring
> 🌐 **Language:** English
> 🎓 **Level:** Graduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

>[!Note]
>
>Please see the course website:
>{%preview https://www.stat.cmu.edu/~siva/teaching/725/ %}

#### Lecture Videos

:::info

**Course Website** ⬇️

{%preview https://scs.hosted.panopto.com/Panopto/Pages/Sessions/List.aspx#folderID=%2235028909-68d2-46be-b0d8-af020148d304%22 %}

:::

### CS439 Optimization for Machine Learning, by EPFL

>[!Tip]
> 🧑‍🏫 **Instructor:** Martin Jaggi and Nicolas Flammarion
> 🏫 **Offering Institution:** École Polytechnique Fédérale de Lausanne
> 📅 **Semester:** 2026 Spring
> 🌐 **Language:** English
> 🎓 **Level:** Graduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

#### Syllabus

>[!Note]
>Convexity, Gradient Methods, Proximal algorithms, Subgradient Methods, Stochastic and Online Variants of mentioned methods, Coordinate Descent, Frank-Wolfe, Accelerated Methods, Primal-Dual context and certificates, Lagrange and Fenchel Duality, Second-Order Methods including Quasi-Newton Methods, Derivative-Free Optimization.

>[!Caution]
>{%preview https://github.com/epfml/OptML_course?tab=readme-ov-file %}

#### Lecture Videos

:::info
**Lecture Videos** ⬇️

{%preview https://mediaspace.epfl.ch/channel/CS-439+Optimization+for+machine+learning/31980 %}

:::

## Other Mathematical Courses

There are many useful math-related courses suitable for AI Ph.D. student. I collect them here.

### AM207 Stochastic Methods for Data Analysis, Inference and Optimization, by Harvard

>[!Tip]
> 🧑‍🏫 **Instructor:** Weiwei Pan
> 🏫 **Offering Institution:** Harvard University
> 📅 **Semester:** 2022
> 🌐 **Language:** English
> 🎓 **Level:** Graduate
> 🎥 **Video Avaliable:** <font color="#DC143C">**YES**</font>

{%preview https://onefishy.github.io/am207/ %}