# Using Physics Informed Neural Networks to solve Differential Equations
`PaAC Open Projects 2024`

## Description:
In this project, we'll learn to use **Physics Informed Neural Networks (PINNs)** to solve Differential Equations. We will familiarize ourselves with the **Pytorch** library and if time permits, **JAX** to build the PINN’s. We will also delve into learning and using **Matlab** for data generation (Consequently, learning how to solve ODE's and PDE's using Matlab).

The project ends with the completion of a *optional* Capstone - Using PINN’s to solve the Non-Linear Schrodinger Equation.

## Timeline
### Week 1 (29th aug - 5th sep)
- Familiarising ourselves with PINNs
- [ ] [So, What is a Physics Informed Neural Network?](https://benmoseley.blog/my-research/so-what-is-a-physics-informed-neural-network/)
- [ ] [**Original Paper**: Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations](https://www.sciencedirect.com/science/article/pii/S0021999118307125)
- [ ] [Matthieu Barreau - Physics-Informed Learning: Using Neural Networks to Solve Differential Equations](https://www.youtube.com/watch?v=R4ZvksarJ1Q)

---
**MTE prep and Examinations**
All the best!

---

### Week 2 (15th sep - 22nd sep)
- What are Neural Networks?
- [ ] [The Essential Main Ideas of Neural Networks
](https://www.youtube.com/watch?v=CqOfi41LfDw&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1&index=3&t=14s)
- [ ] [Gradient Descent](https://www.youtube.com/watch?v=sDv4f4s2SB8&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1&index=5)
- [ ] [Backpropagation 1](https://www.youtube.com/watch?v=iyn2zdALii8&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1&index=7&t=205s) & [2](https://www.youtube.com/watch?v=GKZoOHXGcLo&list=PLblh5JKOoLUIxGDQs4LFFD--41Vzf-ME1&index=8)
#### Submission 1 (Graded)

| Details | Deadline | Marks | How to submit |
| -------- | -------- | -------- | ------ |
| A quiz to test your understanding of Neural Networks. | September 22, 2024 | 5 | Link will be sent |

### Week 3 (23rd sep - 30th sep)
- Learning Matlab for Data Generation!
> **Note**: You can find the data to a couple differential equations [here](https://github.com/maziarraissi/PINNs/tree/master/main/Data), but if you wish to generate your own data, you can either use python if the analytical solution already exists or solve for it numerically using **Matlab**.

- Learning Matlab *(if needed)*
- [ ] [MATLAB Onramp](https://matlabacademy.mathworks.com/details/matlab-onramp/gettingstarted)
- [ ] [Solving Ordinary Differential Equations with MATLAB
](https://matlabacademy.mathworks.com/details/solving-ordinary-differential-equations-with-matlab/odes)
- [ ] [Solving Nonlinear Equations with MATLAB
](https://matlabacademy.mathworks.com/details/solving-nonlinear-equations-with-matlab/rootfinding)

### Week 4&5 (1st oct - 16th oct)
- Implementing NNs using PyTorch (or TensorFlow)
- [ ] [The StatQuest Introduction to PyTorch](https://www.youtube.com/watch?v=FHdlXe1bSe4&t=135s)
- [ ] [Deep Learning With PyTorch - Full Course](https://www.youtube.com/watch?v=c36lUUr864M)
- [ ] [Github repo of pinns-torch library](https://github.com/rezaakb/pinns-torch)
- Understanding optimizers (A few readings)
- [ ] [Gentle Introduction to the Adam Optimization Algorithm for Deep Learning](https://machinelearningmastery.com/adam-optimization-algorithm-for-deep-learning/)
- [ ] [A Gentle Introduction to the BFGS Optimization Algorithm
](https://machinelearningmastery.com/bfgs-optimization-in-python/)

#### Submission 2 (Graded)

| Details | Deadline | Marks | How to submit |
| -------- | -------- | -------- | ------ |
| Implement a normal Neural Network to solve the differential equation and submit the .ipynb notebook with the results. | October 9, 2024 | 25 | Forms link will be sent |
#### Submission 3 (Graded)

| Details | Deadline | Marks | How to submit |
| -------- | -------- | -------- | ------ |
| Now, implement a Physics informed Neural Network to solve the differential equation and submit the .ipynb notebook with the results. | October 16, 2024 | 40 | Forms link will be sent |

### Week 6 (17th oct - 24th oct)
> This is not compulsory, you can explore JAX if you wish to or skip directly to the capstone project depending on time commitments.
- Implementing the same NN using JAX
- Why JAX?
- [ ] [COMPARING PINNS ACROSS FRAMEWORKS: JAX,
TENSORFLOW, AND PYTORCH](https://openreview.net/pdf?id=BPFzolSSrI)
- [ ] [Just In Time (JIT) Compilers - Computerphile](https://www.youtube.com/watch?v=d7KHAVaX_Rs)
- Learning JAX
- [ ] [Introduction to Google JAX for Physics Informed Neural Networks (PINNs)](https://www.youtube.com/watch?v=H1h0wJVnFJk)
- [ ] [Github repo of pinns-jax library](https://github.com/rezaakb/pinns-jax/tree/main)
#### Submission 4 (Ungraded)
This submission is purely for the participant to further their knowledge and skills.
| Details | Deadline | Marks | How to submit |
| -------- | -------- | -------- | ------ |
| Implement the PINN using JAX | October 24, 2024 | 0 | Forms link will be sent |

### Week 7&8 (25th oct - 9th nov)
> This is not compulsory, this is a more advanced problem which will extend your learning but you can also choose to **submit a report** of the equation/(s) previously solved.

**Capstone Project!**
Solve the NLSE using PINNs and present your work in a report to complete this project. You can find the data for this project [here](https://github.com/maziarraissi/PINNs/blob/master/main/Data/NLS.mat).
- Understanding the Non-Linear Schrodinger Equation (NLSE)
- [ ] [Nonlinear Schrödinger equation, Wikipedia](https://en.wikipedia.org/wiki/Nonlinear_Schr%C3%B6dinger_equation)
- [ ] [The Nonlinear Schrödinger Equation, LibreTexts](https://eng.libretexts.org/Bookshelves/Electrical_Engineering/Electro-Optics/Book%3A_Ultrafast_Optics_(Kaertner)/03%3A_Nonlinear_Pulse_Propagation/3.03%3A_The_Nonlinear_Schrodinger_Equation)
- [ ] [Are there any nonlinear Schrödinger equations?, StackExchange](https://physics.stackexchange.com/questions/718059/are-there-any-nonlinear-schr%c3%b6dinger-equations)

#### Submission 5 (Graded)

| Details | Deadline | Marks | How to submit |
| -------- | -------- | -------- | ------ |
| Submit a report (Google Docs) of your project either solving the Differential Equation of you choice or the capstone project if you choose to partake in that. Marks will be the same regardless of the problem chosen. | November 9, 2024 | 30 | Forms link will be sent |