## Project Idea Presentation ### Coding II - Advanced Frameworks <small>[https://hackmd.io/@biowaffeln/r1qFduLB8#/](https://hackmd.io/@biowaffeln/r1qFduLB8#/) </small> Mark Kvetny --- ## Project Idea - An application that runs emotion detection on your face through the webcam in the background. - Whenever it sees you making a happy expression, it takes a photo of your face and a screenshot. - At the end of the day, it creates a collage of all the things you smiled at. --- ## Inspiration - inspired by a piece at the 24/7 exhibition in Somerset House - a guy captured a video of his desktop + webcam and displayed them next to each other---this project adds an AI twist to this idea --- ## MVP - a program that runs on a laptop, performs face classification (using a pre-trained model) and takes photos and screenshots --- ## Extended goals - create a GUI for desktop that controls the program and displays the images nicely - make it run on an Android smartphone --- ## Possible Technology Stacks - OpenFrameworks (ofxFacetracker, will have to research how to make screenshots in OF) - Python (openCV + pre-trained face classification model + pyscreenshot) <small style="margin-top: 20px">(@Mick suggestions on which stack to prefer are welcome!)</small> --- ## Technology Stack Android - Android SDK and Kotlin - [ML Kit](https://developers.google.com/ml-kit/vision) (includes an on-device API for emotion classification) - Camera API (video capturing) + MediaProjection API (screenshots) --- ## Possible Challenges - with OF it might be trickier to create a GUI and take screenshots - with python there might be performance considerations - with Android it might be difficult to run video capturing in background (must be accessed through a [foreground service](https://developer.android.com/guide/topics/media/camera#considerations)) --- ## Learning Goals - gain familiarity with using AI models - deepen knowledge of either OF or python - learn how to develop an Android App (and learn Kotlin)