# **meeting 10/24** **Advisor: Prof. Chih-Yu Wang \ Presenter: Shao-Heng Chen \ Date: Oct 24, 2023** <!-- Chih-Yu Wang --> <!-- Wei-Ho Chung --> ## **Current Progress** Here are some of the things I've been up to for the past week. ### **Bugs fixing** The reason why the Agent (```torch``` version) cannot learn anything is partly because I didn't actually update the phase matrix due to this bug. ![](https://hackmd.io/_uploads/HJrx6cyGa.png) ![](https://hackmd.io/_uploads/ryu18sEf6.png) ### **Training results** Learning script ![](https://hackmd.io/_uploads/BJHkyZ4MT.png) Independent variables: ```Ns``` and ```model```. #### **PPO with different Ns = {4, 9, 16, 25}** Orange: ```Ns = 4```, Red: ```Ns = 9```, Blue: ```Ns = 16```, Cyan: ```Ns = 25``` ![](https://hackmd.io/_uploads/By-WdlVMp.png) #### **A2C with different Ns = {4, 9, 16, 25}** Orange: ```Ns = 4```, Blue: ```Ns = 9```, Red: ```Ns = 16```, Cyan: ```Ns = 25``` ![](https://hackmd.io/_uploads/BJV-Fe4Mp.png) #### **DQN with different Ns = {4, 9, 16, 25}** Orange: ```Ns = 4```, Blue: ```Ns = 9```, Red: ```Ns = 16```, Cyan: ```Ns = 25``` ![](https://hackmd.io/_uploads/H1fjOeEGT.png) #### **Compare differenct models with their best performance** Pink: ```PPO``` with ```Ns = 4```, Green: ```A2C``` with ```Ns = 4```, Red: ```DQN``` with ```Ns = 9``` ![](https://hackmd.io/_uploads/SkxUqZ4za.png) #### **Increase the number of users to Nk = 2** Green: ```A2C``` with ```Nk = 1, Ns = 4``` Pink: ```A2C``` with ```Nk = 2, Ns = 4``` Red: ```PPO``` with ```Nk = 2, Ns = 4``` Cyan: ```DQN``` with ```Nk = 2, Ns = 4``` ![](https://hackmd.io/_uploads/rkVY5bVMp.png) #### **Compare different numbers of users Nk = {1, 2, 3, 4}** Blue: ```DQN``` with ```Nk = 1, Ns = 4``` Cyan: ```DQN``` with ```Nk = 2, Ns = 4``` Green: ```DQN``` with ```Nk = 3, Ns = 4``` Gray: ```DQN``` with ```Nk = 4, Ns = 4``` Orange: ```DQN``` with ```Nk = 5, Ns = 4``` ![](https://hackmd.io/_uploads/rkYiMsVfT.png) #### **Compare the complexity of different settings** 1. ```Nk = Nt = 3, Ns = 4, obs shape: (78, )``` 2. ```Nk = Nt = 2, Ns = 4, obs shape: (52, )``` - ```DQN``` > ```PPO``` > ```A2C``` 3. ```Nk = Nt = 1, Ns = 4, obs shape: (30, )``` 4. ```Nk = Nt = 1, Ns = 9, obs shape: (65, )``` - except for the case of ```DQN``` 5. ```Nk = Nt = 1, Ns = 16, obs shape: (114, )``` 6. ```Nk = Nt = 1, Ns = 25, obs shape: (177,)``` - except for the case of ```DQN``` ### **Insights** 1. The larger the number of RIS elements ```Ns```, the higher the resulting MSE and the lower the ```reward```. 2. Adding more users seems to benefit our task. ### **Future works** 1. Try to do inference, see if we can actually extract and reproduce the results. 2. Try reducing the number of bits (```bits = {1, 2}```) and observe its impact on the resulting MSE. - Though reducing the number of bits doesn't affect the complexity of the observations. 3. Try to reduce the given informations and lower the dimension of ```observations``` 4. Try to repeat all these experiments on ```Box discrete``` case. ![](https://hackmd.io/_uploads/r1uhvtDGp.png)