# Gesture controler for streming services.

Matacuta Alexandra_Cristina cm222zr
Time:12h
Price:600SEK

# Objective
Watching series is my favourite thing to do, but watching series and eating crips is heaven. But I have a problem: my keyboard and my touchpad get always dirty if I am trying to stop the video or to increese the volume. By creating a gesture controler device I can enjoy my crips, watch my serie and keep my keyboard clean with just one simple movement.

# Material

| Name | Function | Price|
| -------- | -------- | -------- |
|Arduino Uno | Microcontroller | [in bundle](https://www.bitmi.ro/kit-arduino-uno-r3-ch340-10171.html?gclid=CjwKCAjwi8iXBhBeEiwAKbUofa-9h0kEbcjh48Gi0VSPZtNW-WkrABPVNrAyrbDsnn21ONs8DH-8yxoCd1wQAvD_BwE) |
| Ultrsonic Sensor | Detects the motion | [in bundle](https://www.bitmi.ro/kit-arduino-uno-r3-ch340-10171.html?gclid=CjwKCAjwi8iXBhBeEiwAKbUofa-9h0kEbcjh48Gi0VSPZtNW-WkrABPVNrAyrbDsnn21ONs8DH-8yxoCd1wQAvD_BwE)|
|Jumper wires | Conect the sensors to the mictrocontroller | [in bundle](https://www.bitmi.ro/kit-arduino-uno-r3-ch340-10171.html?gclid=CjwKCAjwi8iXBhBeEiwAKbUofa-9h0kEbcjh48Gi0VSPZtNW-WkrABPVNrAyrbDsnn21ONs8DH-8yxoCd1wQAvD_BwE) |

In this project I choose to work with Arduino Uno since it is realively easy to use, does not cost a lot, and can be used further if I think in making the project more complex.

# Computer set up

First thing first you need to instal Python IDLE on your computer/laptop. That can be easly done by entering the Python website, downloading it and following the steps shown on the screen.

After instaling Python you need to instal the PySerial Library which is used to read and write serial data to the microcontroller. For Windows which is what I have you just need to follow the below [link](https://pypi.org/project/pyserial/2.7/).
For computers operating on windows, the downloaded file will be a zip file that needs to be extracted.Once the file was extracted we need to copy the "link" of the file.

After copying it, open the Command Prompt on your computer and we are going to type in: ***cd THE PATH WE COPIED*** and press enter. By doing this we basically open the folder. The next comand will be:***python setup.py install***. This command should install the library.

The last step is instaling the **Pyautogui library.** This package has the ability to simulate mouse cursor moves and clicks as well as keyboard button presses.
After instaling Python on our computer, we open Command Prompt and type in the command:***pip install pyautogui*** and press enter. By doing so the system will download and install the package from the source.
# Putting everything together

Now we go to the fun part, connecting the sensors to Arduino and to the computer. Since the project is prety easy and budget friendly I think connecting the sensors is the easiest part.
Let's start with the Ultrasonic Sensors. The one we will place on the left side of the screen will have the Trriger and Echo pins to the Pins 11 and 12 of the Arduino. For the one that will be placed on the right side, the Trigger and Echo pins will be connected to Pins 6 and 5 of the Arduino. The sensors are powered by the on board Voltage regulator of Arduino. We will use an USB to connect Arduino to the computer thus the Arduino commands will be sent through it to the computer. This data will be read by python and actions will be performed.

After connecting all the sensors we will place the sensors on the side of the computer using some tape.

# The code
#### Arduino

For this project we need Arduino to be programmed to read the distance of the hand in regards to the sensors. Different gestures and distances can give different outputs, I wrote the code for 3 different gestures.
1. Hand in front of the Right Sensorat a particular far distance =>Rewind
2. Hand in front of the Left Sensor at a particular far distance =>Foward
3. Both hands in front of the sensor between 40 and 50 cm=>Play/Pause Video.

We will start by defining the I/O pins.

const int trigger1= 11; //Trigger pin of the 1st sensor
const int echo1= 12; //Echo pin of 1 sensor
const int trigger2= 6; // Trigger pin  of the 2nd sensor
const int echo2=5; // Echo pin of the 2nd sensor
void setup() {
Serial.begin(9600);

pinMode (trigger1, OUTPUT);
pinMode (echo1, INPUT);
pinMode (trigger2, OUTPUT);
pinMode (echo2, INPUT); 
}

We will calculate the distance between the Sensor and the hand each time before concluding on any action. We have written a function named *calculate_distance()* which will return us the distance.

/*###Function to calculate distance###*/
void calculate_distance@(int trigger, int echo)
{
digitallWrite(trigger, LOW);
delayMicroseconds(2);
digitlalWrite(trigger, HIGH);
delayMicroseconds(10);
digitalWrite(trigger, LOW);

time_taken=pulseIn(echo, HIGH);
dist=time_taken*0.034/2;
if(dist>50);
dist=50;
}


calculate_distance(trigger1,echo1);
distL =dist; //get distance of left sensor

calculate_distance(trigger2,echo2);
distR =dist; //get distance of right sensor

Now we will continue with the hand gesture:

if ((distL >40 && distR>40) && (distL <50 && distR<50)) //Detect both hands
{Serial.println("Play/Pause"); delay (500);}


if ((distL >40 && distL<50) && (distR ==50)) //Detect Left Hand
{Serial.println("Rewind"); delay (500);}

if ((distR >40 && distR<50) && (distL ==50)) //Detect Right Hand
{Serial.println("Forward"); delay (500);}

#### Python Programming

Since we have have already installed the pyautogui module and pyserial module we will now import them and import time.

import serial #Serial imported for Serial communication
import time #Required to use delay functions
import pyautogui

The connection between Arduino and the computer is made via COM. Use device manager to find to which COM port your Arduino is connected to.

ArduinoSerial = serial.Serial('com4',9600) #Create Serial port object called arduinoSerialData
time.sleep(2) #wait for 2 seconds for the communication to get established

while 1:
    incoming = str (ArduinoSerial.readline()) #read the serial data and print it as line
    print incoming
    
    if 'Play/Pause' in incoming:
        pyautogui.typewrite(['space'], 0.2)

    if 'Rewind' in incoming:
        pyautogui.hotkey('ctrl', 'left')  

    if 'Forward' in incoming:
        pyautogui.hotkey('ctrl', 'right') 

# Conclution

Having my dream come true, never felt better, especially because I always press on FORWARD when watching a movie.For me it was pretty difficult in the begining, since I have never coded/make a device. If I would start again I would not change much, maybe add more features.