Monitor student attention in class using Computer Vision
===

###### tags: `PoseNet` `computer science` ‘teacher’ ‘student’

:::info
- **What:** What is computer Version
- **Who:** Who are we focus on when use it?
- **How:** How to analyse the result?
- **bias:** How to make sure our model is not biased?
- **overfitting:** How to make sure our model will not overfit?
- **Misuse:** What are the possible misuses of this application?
- **Data leakage:** Can we open source our model and dataset? Are there any sensitive data in the training dataset?
- **Hacking:** What would happen if someone found a way to trick the model?
:::

## What is computer Version

For a computer, an image (or a video), is just a series of numbers each representing its pixel color. Using Computer Vision, they can try to make sense of these numbers and start detecting objects, shapes, people, roads, etc.

## Who are we focus on when use it?

Student attention and teacher attractiveness

## How to analyse the result?

Computer version can locate some points and return the quad values of the data to the system. Data can be derived by analyzing these values. In later analysis, we will use poseNet to explain and poseNet can monitor the following 17 points：

---
- nose
- leftEye
- rightEye
- leftEar
- rightEar
- leftShoulder
- rightShoulder
- leftElbow
- rightElbow
- leftWrist
- rightWrist
- leftHip
- rightHip
- leftKnee
- rightKnee
- leftAnkle
- rightAnkle
---
For example,we can use the following characteristics to analyze whether students pay attention to listening
- Whether to play with a mobile phone(judgment criteria: the mobile phone appears and the monitored object bows their heads)
- Whether to be in a daze (judgment criteria: other students look up and the monitored object bows their heads)
- Whether to chat with others (judgment criteria: the horizontal distance between the left and right eyes is much smaller than the vertical distance from the eyes to the nose)
use the following characteristics to analyze whether teacher is attractive
- The level of concentration of the students while listening to the lecture

## How to make sure our model is not biased?
---
- Sometimes students bow their heads to do their homework, and hold their mobile phones to check information. There should be some parameters to determine whether the teacher has issued a command to bow his head or look at the material. It is better posenet should be combined with other voice software for detection.
- We can rank the attention of classmates and the attractiveness of teachers through manual observation and voting. It is then compared to the data given by the system, and the logic is adjusted to bring the results closer to the artificial ones. However, not all artificial results can be used as parameters, and only 100% reliable data are selected.

## How to make sure our model will not overfit?
Do not use parameters that are not 100% standard as reference items. For example, sometimes the students sitting in the front row will listen more carefully than the students sitting in the back row, but the system should not take the seat information as a parameter.

## What are the possible misuses of this application?
If schools give rankings and bonuses based on the attractiveness of teachers in the application, some teachers may be able to cater to this monitoring system. For example, students are required to put away their mobile phones(as mobile phone appearance is a standard about not pay attention), and if they are found, they will be severely punished. Just to avoid some behaviors to get a high score, not to change for the sake of teaching a good class

## Can we open source our model and dataset? Are there any sensitive data in the training dataset?
Don't make the dataset public because there are many minors among the students. If their faces are obtained by criminals, they may do something harmful to minors. At the same time, if it is made public, it needs to be discussed with students in advance. Some students are considering that a camera has been recording themselves, and that this data may be used by others in the future, which may cause pressure and affect their learning.

## What would happen if someone found a way to trick the model?
Neither the student's attention nor the teacher's attractiveness calculated by the system is not accurate.If schools using this software plan to improve teaching methods based on this data, the teaching results will be worse.