# **meeting 10/24**
**Advisor: Prof. Chih-Yu Wang \
Presenter: Shao-Heng Chen \
Date: Oct 24, 2023**
<!-- Chih-Yu Wang -->
<!-- Wei-Ho Chung -->
## **Current Progress**
Here are some of the things I've been up to for the past week.
### **Bugs fixing**
The reason why the Agent (```torch``` version) cannot learn anything is partly because I didn't actually update the phase matrix due to this bug.


### **Training results**
Learning script

Independent variables: ```Ns``` and ```model```.
#### **PPO with different Ns = {4, 9, 16, 25}**
Orange: ```Ns = 4```, Red: ```Ns = 9```, Blue: ```Ns = 16```, Cyan: ```Ns = 25```

#### **A2C with different Ns = {4, 9, 16, 25}**
Orange: ```Ns = 4```, Blue: ```Ns = 9```, Red: ```Ns = 16```, Cyan: ```Ns = 25```

#### **DQN with different Ns = {4, 9, 16, 25}**
Orange: ```Ns = 4```, Blue: ```Ns = 9```, Red: ```Ns = 16```, Cyan: ```Ns = 25```

#### **Compare differenct models with their best performance**
Pink: ```PPO``` with ```Ns = 4```, Green: ```A2C``` with ```Ns = 4```, Red: ```DQN``` with ```Ns = 9```

#### **Increase the number of users to Nk = 2**
Green: ```A2C``` with ```Nk = 1, Ns = 4```
Pink: ```A2C``` with ```Nk = 2, Ns = 4```
Red: ```PPO``` with ```Nk = 2, Ns = 4```
Cyan: ```DQN``` with ```Nk = 2, Ns = 4```

#### **Compare different numbers of users Nk = {1, 2, 3, 4}**
Blue: ```DQN``` with ```Nk = 1, Ns = 4```
Cyan: ```DQN``` with ```Nk = 2, Ns = 4```
Green: ```DQN``` with ```Nk = 3, Ns = 4```
Gray: ```DQN``` with ```Nk = 4, Ns = 4```
Orange: ```DQN``` with ```Nk = 5, Ns = 4```

#### **Compare the complexity of different settings**
1. ```Nk = Nt = 3, Ns = 4, obs shape: (78, )```
2. ```Nk = Nt = 2, Ns = 4, obs shape: (52, )```
- ```DQN``` > ```PPO``` > ```A2C```
3. ```Nk = Nt = 1, Ns = 4, obs shape: (30, )```
4. ```Nk = Nt = 1, Ns = 9, obs shape: (65, )```
- except for the case of ```DQN```
5. ```Nk = Nt = 1, Ns = 16, obs shape: (114, )```
6. ```Nk = Nt = 1, Ns = 25, obs shape: (177,)```
- except for the case of ```DQN```
### **Insights**
1. The larger the number of RIS elements ```Ns```, the higher the resulting MSE and the lower the ```reward```.
2. Adding more users seems to benefit our task.
### **Future works**
1. Try to do inference, see if we can actually extract and reproduce the results.
2. Try reducing the number of bits (```bits = {1, 2}```) and observe its impact on the resulting MSE.
- Though reducing the number of bits doesn't affect the complexity of the observations.
3. Try to reduce the given informations and lower the dimension of ```observations```
4. Try to repeat all these experiments on ```Box discrete``` case.
