# KIMIA Home Neck & Spine measurement documentation

###### tags: `Kinexcs`

## Overview

| Features | Links |
| ----------------- |:----------------------- |
| GitHub Repository | [kinexcs/CV-Therapist](https://github.com/kinexcs/CV-Therapist) |
| KIMIA Home's Overview | [Kinexcs Digital Therapy](/p17yQf93RqKa5PpQ_m_KGg) |
| CV Guide | [Kinexcs CV Guide (19/08/21)](/EniHpJEfTXakbbvoe0PVjQ) |

## :memo: Structure

- Below is the directory of the neck spine measurement. It contains 6 main files. `<asset_path.py>` `<custom_state_machine.py>` `<state_machine_controller.py>` `<generate_graphs.py>` `<generate_report.py>` `<setup.py>`


### 1. `<asset_path.py>`

- This file is currently untouched. It can be utilised if assets are to be added in the future.

### 2. `<custom_state_machine.py>`
- This file contains the functions and conditions that the program requires.


- For the measurement programs, flags will be mainly utilised to determine the satisfactory of a condition. Other programs may require more than just flags for conditions.
- The flags can be renamed accordingly to satisfy readibility, extensibility and also certain naming conventions.


- The class ==<CustomStateMachine(HierarchicalMachine)>== is imported from the *<transitions>* library and will be used for the finite state machine methodology in the program.


### 3. `<state_machine_controller.py>`

- This file will mainly include the use of the finite state machine, as well as the drawings of the mediapipe landmarks.
- The class *<FSMController>* is initalised first.


- The states and transitions of the finite state machine are declared in the following manner.


- For each state to transition to another state, the condition under the specific trigger has to be satisfied.
- The *trigger* is a name to determine a specific transition.
- The *source* is the state that precedes the *dest* state.
- The *conditions* includes the function that is required for the state to transition.

- The function *<run_state_machine>* is used to run the state machine. It takes in the parameters from ==<setup.py>==`


- The joints or landmarks that are being utilised from mediapipe can be drawn in this manner. *<ignore_idx>* disables the display of specified landmarks.


- What happens during each state is then determined by an if statement followed by the *<curr_state>* (current state)


- The function *<check_state>* is used to determine the current state as well as going through the triggers of specific states.


- The function *<report>* outputs the required values into a csv file.


### 4. `<setup.py>`
- This file is the **'main file'** of the directory, and the software can be run from here. The computer's camera is being utilised here through cv2, which will output the video capture from the camera.


- The measurement function is also in this file. For the neck spine measurement, the mediapipe Holistic model is used. This allows the usage of mediapipe models such as the FaceMesh, BlazePose and HandPose. More details can be explored at [MediaPipe Holistic]([https://google.github.io/mediapipe/solutions/holistic.html](https://)).


- Different measurements will require different angles and coordinates to be obtained.
- For the ==Lateral Spine Flexion== measurement, the mid points of the shoulders, hips and knees are used to determine the range of motion.


- For the ==Lateral Neck Flexion and Extension== measurement, a line is drawn from the mid points of the shoulders to the nose, to achieve the range of motion.


- For the ==Neck Flexion and Extension== measurement, the ear, shoulder and hip is used to calculate the angle. A *well-trained model* is required to obtain the range of motion of the neck flexion and extension. This is yet to be implemented. More details of machine learning models can be found at [scikit-learn]([https://scikit-learn.org/stable/modules/neural_networks_supervised.html](https://)).


- For the ==Neck Rotation== measurement, there are 2 different methods that are currently implemented: **BlazePose** and **FaceMesh**.
- BlazePose: The left outer eye and right outer eye are used to determine the coordinates.

- FaceMesh: Points 226 and 446 are used to calculate the coordinates.


### 5. `Uploading data to s3`
- After every assessment, when navigated to the last result page,the program would automatically upload the json file to s3 bucket "cv-therapist-json".
- The screen capture video would be uploaded when the user exit the program via 'ESC' or restart the measurment via 'r'. Recorded video would be uploaded to s3 bucket "cv-therapist-video".

### 6. `How to make EXE`
- To convert the application to EXE type the following,

`pyinstaller testspine.spec`

- If encountered any issues regarding imports, enter the library with issues into the 'Hidden Import' in testspine.spec.
- Take note, pyinstaller only works on Windows.
- When importing new libraries, make sure to pip install in the virtual environment as the EXE paths for libraries are in the venv.

## Conclusion
Authors: Benjamin, Zijian

Good Luck!

<!-- - Auto-generated Table of Content
[ToC]
-->