## RL Mario Agent > VLG Open Project Summers 2022 ### Problem Statement Playing and winning a computer game is a very challenging and rewarding task. People have always tried to develop "smart players" who can learn the game and then ace it. This project aims to build a robust RL agent that can make it through the first level of Super Mario Bros. ### Motivation When it comes to solving problems where you learn from the environment by taking actions and getting rewards, reinforcement learning algorithms are a very intuitive choice. Playing the game of Mario is a multi-action task, where you have to keep avoiding obstacles and earn points to survive till the end. ### Solution - To play and ace a game is to maximize your reward and stay alive till the end of the mission. This type of problem can be thought of as a Q-function learning task - A state-action value is the quality of being in a particular state and taking a specific action off of that state - If an agent is on a particular state S, it will take action a from s such that Q(s, a) has the highest value, i.e., it will try to go to the state from which it can expect the highest reward. But to do this, we need to learn the Q-values of all the state-action pairs - Use the Bellman Updation Equation for Q-Learning - There is one major issue with Q-learning that we need to deal with: over-estimation bias, which means that the Q-values learned are higher than they should be. To get more accurate Q-values, we use something called double Q-learning. - So we go for DQN. In double Q-learning, we have two Q-tables: one which we use for taking actions, and another specifically for use in the Q-update equation ### Timeline #### Week 1 : `May 25, 2022` - `May 31, 2022` - Get comfortable with Google Colab and pytorch framework - Understand basic concepts of RL - Basic Jargon - Bellman Equations - Q learning - Get familiar with super mario gym #### Week 2 : `Jun 1, 2022` - `Jun 7, 2022` - Understand the utility functions to display frames and buffers while training - Use the Q-Learning approach to solve the problem - Write `Q-update` rules - Make a pipeline fitting all these elements together and run your first training loop - Analyse the results via graphs #### Week 3 : `Jun 8, 2022` - `Jun 14, 2022` - Get yourselves familiar with Double Q-Learning - Maintaining two Q-value tables - Write a DQN Agent - Update the `Q-update` rules - Write a `DQN Solver` to make use of deep Q-Learning - Write the `remember`, `recall`, `experience_replay` methods - Train the DQN model - Run inference using the fine-tuned model - Analyse with graphs and write a conclusion ### Resources - [Great overview on RL](https://lilianweng.github.io/posts/2018-02-19-rl-overview/) - [Bellman Equations](https://www.analyticsvidhya.com/blog/2021/02/understanding-the-bellman-optimality-equation-in-reinforcement-learning/) - [Implementation of Reinforcement Learning Algorithms](https://github.com/dennybritz/reinforcement-learning) - [Q-Learning](https://blog.floydhub.com/an-introduction-to-q-learning-reinforcement-learning/) - [Double Q-Learning](https://towardsdatascience.com/deep-double-q-learning-7fca410b193a) - [Doubel Q-Learning with Tensorflow](https://rubikscode.net/2021/07/20/introduction-to-double-q-learning/)
Last changed by  
 
ChanBong
1756

Read more

Open Source contributions by C

I contributed in PyTorch Docathon H2 2023. I was second on the leaderboard for my contributions

2/14/2024
MATHS CLUB IITR

To promote r

12/4/2023
Untitled

Primary Team ID: 17 Secondary Team ID: 34 Abstract Vehicle routing is an important problem in the logistics industry, particularly for last-mile hubs. These hubs are responsible for delivering goods to customers in a specific area, and they need to optimize their routes in order to minimize costs and maximize efficiency. One key aspect of vehicle routing is capturing the accurate weight of the goods to be delivered. This is important because it affects the amount of fuel required for each trip, as well as the capacity of the vehicle. Additionally, if the weight of the goods is not accurately captured, it can lead to delays and other issues in the delivery process This report summerises our approach to solving the dyanamic varient of the vehicle routing problem with pickups and deliveries considered in the Inter IIT Tech Meet 11.0. In what follows, we will describe our appraoch to tackle this problem with techniques from Operatino Reasech and Machine learning. Our current solver is a mixture of the most succesful experiments we have run so far. We also showcase current web and app framework and design. We highlight our appraoch and setup to accuretly predict voluemteric and dead weight of the items provided. Methodology and Approach Volume Estimation from images

1/18/2023
RL

Why NAS NAS is a technique for finding the best possible architechture for neural nets. It takes off the human bias and finds the best archi suited for out needs. What NAS Search Space Types of layer we can experiment with, number of layers, how will they we connected. Often biased as we are the ones to inclue these choices

11/30/2022
Read more from ChanBong
Published on HackMD