Notes on "[Sentio: Driver-in-the-Loop Forward Collision Warning Using Multisample Reinforcement Learning](https://dl.acm.org/doi/pdf/10.1145/3274783.3274843)"

# Notes on "[Sentio: Driver-in-the-Loop Forward Collision Warning Using Multisample Reinforcement Learning](https://dl.acm.org/doi/pdf/10.1145/3274783.3274843)" ###### tags: `ADAS` `Sentio` ## Problem setting: The authors propose "Sentio", a Reinforcement Learning based algorithm to enhance the Forward Collision Warning (FCW) system leading to Driver-in-the-Loop FCW system. On top of considerating the threshold of time-to-crash by traditional FCW systems this algo also claims to take in account 1) Driver's preference or mood. 2) Change in the driver's mood over time. ## Aproach: To address the above challenges, Sentio: 1) Create a discrete state space MDP by quantising vehicle sensing time, relative velocity and relative distance. 2) Propose multi sample Q-learning, improves upon standard Q-learning in FCW context. 3) Monitor Driver's response and state of car continously to provide "personalised" warnings. ## Prior challenges: 1) Dynamic and time-varying rewards: Rewards are related to preferred relative distance/velocity of the driver, which can change over time. 2) Ignoring actions generated by the RL agent. 3) Reward time horizon: Our rewards are related to whether the driver follows our FCW or not. But human response time is neither instantaneous (few seconds) nor fixed.