---
title: 'Project documentation template'
disqus: hackmd
---

STABBY: Smart protecTive and Assistive Ballistic Building sYstem
===

## Table of Contents

[TOC]

## Project Overview
Smart home assistants like Alexa and Google Home have been growing in popularity over the last decade. To explore how home assistants work and expand on their capabilities, we will implement our own, dorm-friendly smart assistant called STABBY (Smart proTective and Assistive Ballistic Building System).

STABBY is a smart home system with voice assistance and surveillance capabilities. It can tell you the weather, control your lights, give you daily trivia questions, play your favorite music, and have a fun (scripted) chat with you. STABBY's security system keeps your building safe; it contains a security camera with facial recognition and a gesture-based password for entry. By visiting a webpage on our server, you can monitor your home at all times. If you're away and an unrecognized intruder enters your home, STABBY will shoot ballistics from a turret mounted at your door--a novel defensive measure that we have not yet seen in existing home assistants.

Intended Functionality
---

STABBY has two main functionalities: security and assistance. For security, STABBY will detect if someone has entered your room using the ultrasonic sensors to monitor if the door has been opened and there is motion in the room. If motion occurs, STABBY will check to see if the secure gesture has been made. If not, it will turn on the camera, take a picture of whatever has entered the room, and send the photo up to the server to check if the person is known and welcome or an intruder. If the secure gesture is made or the photo taken is of a known person, STABBY will welcome them into the room and log their entrance into the database. In the case that the person who entered is unknown, STABBY will issue a warning for them to leave the room. If motion is still detected STABBY will fire a turret at the intruder. Using the database, there will also be a website which displays who has recently entered the room or if an intruder was detected. One of the ultrasonic sensors will always be on so that STABBY can recognize motion, but the camera will remain off unless motion is detected and gesture recognition fails in order to conserve power.

On the assistance side, STABBY will be voice activated by the phrase "hey STABBY". Once it registers that someone has said that it will listen for other commands such as "what time is it?" or "turn on the lights" which it will then complete by either responding verbally or doing the task assigned. If STABBY doesn't register a command it will prompt the user to regive the command. The current list of planned actions STABBY will be able to assist with is: controlling the lights (LEDS), giving the time, weather, pressure, or temperature, asking trivia questions, and reseting the secure gesture for the security system. We will likely add more commands and funcionalites to STABBY later on.

System Block Diagram:

State Machine:

Task List and Technical Challenges
---
Some technical challenges we expect to face:

**Wiring:** Wiring the board such that all components are functioning.
**Motion Detection:** Detecting a person in front of the door.
**Facial Recognition:** Recognizing faces in frame, and recognizing faces that are within the database
**Gesture Recognition:** Recognizing a set of gestures that adds a face into a database of recognized faces
**Faces and Functionality Database:** Storing and accessing recognized faces
**Voice Command/Recognition:** Having a set of commands, and recognizing said commands that are spoken.
**Device Speaking:** Device should be able to report to user using sound.
**Information API:** Returns requested information when asked.
**Temperature/Pressure Sensor:** Being able to report and store the temperature and pressure readings gathered from the sensors.
**Markdeep Document:** Documenting the project progress, as well as documentation for system and code.
**Code Management:** Organizing group's code and git branches.
**Money Management:** Organizing purchase of parts needed and any replacements.
**Integration:** Integrating all parts from each group member into one cohesive project.
**Intruder Management:** System that deters the intruder from continuing.
**Miscellanious Jobs:** Anything that pops up during the project that is unaccounted for in this list.

This list is subject to change at any time.

Work Split
--
Here is the assignment of the tasks above to each person. The assignment is subject to change.

**Bianca**: Motion detection, gesture recognition, money/materials management.
**Warren**: Speaker, voice command recognition, code manager.
**Isabella**: Facial recognition, face database
**Isabel**: Alerts: LED, Stabby, and other future features related to combating intruders.
**Alicia**: Information API, temperature/pressure sensors.

Wiring, integration: **EVERYONE!!**

Parts List
---

**Camera**
Description: Arducam Mini Module Camera Shield with OV2640 2 Megapixels Lens for Arduino UNO Mega2560 Board
Item number: 1
Vendor: Amazon https://www.amazon.com/Arducam-Module-Megapixels-Arduino-Mega2560/dp/B012UXNDOY/ref=sr_1_4?dchild=1&keywords=arducam&qid=1617653366&sr=8-4
Price: $25.99

**Ultrasonic sensors**
Description: Excelity 3pcs Ultrasonic Module HC-SR04 Distance Sensor with 3pcs Mounting Bracket
Item number: 6
Vendor: Amazon https://www.amazon.com/Excelity-Ultrasonic-HC-SR04-Distance-Mounting/dp/B07SC1YJ21
Price: $7.99*2 = $15.98

**LEDS**
Description: LEDS
Item number: 10
Vendor: Joe Steinmeyer
Price: 0

On Campus Resources
---
Metropolis 3d printers/laser cutters
6.08 classroom materials (LEDs, wires, misc electronic components as we need them)

Project Timeline
---
Milestone:
| Date | Bianca | Warren | Isabel | Isabella| Alicia |
|------|--------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------|
| 4/22 | Project Proposal | Project Proposal | Project Proposal| Project Proposal | Project Proposal |
| 4/29 | Order/distribute all of the necessary parts Read up on documentation for ultrasonic sensors and write the code for motion detection | Develop scripts to make sounds/noise based on the decision made by the facial/motion recognition module about the person at the door. | Set up different modes of custom LED lighting with the light strip. Also acquire and set up motors and get them working. | Get facial recognition code to recognize familiar/unfamiliar faces. Create a database on the server to record familiar vs. unfamiliar faces. | Create framework for API requests, as well as data gathering and creating a database to store responses |
| 5/6 | Create code so that the ultrasonic sensors can recognize gestures and have code to reset/reassign what the secure gesture is | Work on the connection between outputs and usable data for the speaker. | Integrate LED lighting with the voice commands and other controls. Assemble motors & other components for stabby/juice, get stabby to start stabbing. | Integrate the camera onto the breadboard and have it send and receive information about the images successfully | Integrate requests to respond to the correct voice commands |
| 5/13 | Integrate motion detection and gesture recognition to trigger the facial recognition | Follow up with updates from motion/gesture detection and generate sounds for different situations. | Finish up stabby. Integrate stabby with the rest of system + controls. | Integrate facial recognition to only activate after the gesture recognition has failed and to activate intruder alert if facial recognition fails | Finish integrating functions with rest of project |
| 5/20 | Last minute adjustments and final report | Last minute adjustments and final report | Last minute adjustments and final report | Last minute adjustments and final report | Last minute adjustments and final report |

Demo:
| Date | Bianca | Warren | Isabel | Isabella | Alicia |
|---------------------------------------|-----------------------------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------|
| 4/22 | Project Proposal | Project Proposal | Project Proposal | Project Proposal | Project Proposal |
| 4/29 (assuming parts come in on time) | Show a working motion detection system by having sensor detect a hand moving towards it | Show an operating speaker that produces welcoming/warning noise. | Manually show the different lighting modes for each situation. Also show the motors being controlled. | Show face on camera (computer webcam) and have the code recognize it and send it to a database. Can be done locally, just gotta show that the code works | Show that the requests return correct data, and that the results are stored correctly |
| 5/6 | Gesture at the sensor and have it recognize a predesignated gesture vs a different one | Show a working speaker integrated with the recognition system could warn strangers or welcome known faces. | Show the LEDs being controlled by voice/other events (like an intruder alert), and functioning turret part of stabby. | Have arduino camera send a photo to the server and receive information about the face, e.g. if it’s in the database or not | Show requests sent correctly when integrating with vocal commands |
| 5/13 | Show that camera only activates and sends/receives data after motion detection occurs | Show an updated warning system integrating all functionalities. | Show stabby reacting to a break in and successfully intimidating the intruder. | Show that the camera only activates and sends/receives data after motion detection occurs. | Show that functions are retained when integrated into the rest of the system, and do not interfere with other parts |
| 5/20 | Working final project | Working final project | Working final project | Working final project | Working final project |