---
# System prepended metadata

title: 隱形滑鼠開發日誌

---

# 隱形滑鼠開發日誌
- [name=Yu-Cheng Lin]
- [time=Sun, Jun 2, 2023 11:50 AM] 
:::info
老兄早安，這裡是老兄們的隱形滑鼠開發園地，有加寫東西記得署名

:::
老兄先把報告要用的資訊打上來了，反正到時候會忘就像：
```
import brain
if not 先寫下來：
    brain.forget(報告要用的)
    return 蛤
```

### 研究動機
因為電腦教室的軌跡球~~難用~~使用上有些不順，帶滑鼠又頗麻煩，我們決定帶個程式來當滑鼠
### mediapipe
![手部辨識節點編號](https://hackmd.io/_uploads/rypiSCOI2.png)
手部辨識節點編號


## 6/2 隱形滑鼠開發 Day1
基本上拍得到手，接下來要搞懂如何取得 landmarklist的 x y座標
https://hackmd.io/m7Y3tO3KRUyb2GfARoBHWA#
:::spoiler
```py=
import mediapipe as mp
import cv2
import pyautogui as pg

Cam = cv2.VideoCapture(0)

mp_hands = mp.solutions.hands
mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
with mp_hands.Hands(
    model_complexity=0,
    min_tracking_confidence=0.5,
    min_detection_confidence=0.5) as hands:

    while True:
        rec, img = Cam.read()
        if not rec:
            print('BRUH')
            break
        img = cv2.resize(img, (800,600))
        img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
        results = hands.process(img2)
        if results.multi_hand_landmarks:
            for hand_landmarks in results.multi_hand_landmarks:
                mp_drawing.draw_landmarks(img,
                                          hand_landmarks,
                                          mp_hands.HAND_CONNECTIONS,
                                          mp_drawing_styles.get_default_hand_landmarks_style(),
                                          mp_drawing_styles.get_default_hand_connections_style())

                pg.moveTo(hand_landmarks)
        cv2.imshow('BRUH', img)
        if cv2.waitKey(5)==ord('q'):
            break
Cam.release()
cv2.destroyAllWindows()

```
:::

今天不錯，上半天，放假囉
*by ELO*

---

## 6/3 Day2

#### 22:32 
老兄，可以動了
不過這個跟我們真的要的不太一樣，我設定成根據食指指尖座標移動至螢幕的對應位置
這樣設定會有兩個問題：
1. 手移動到邊邊時，模型偵測不到，進而無法移動鼠標到底邊
2. 用起來挺智障的，要一直注意相機拍得到的區域，使用上不自然

#### 22:40
老兄，可以點擊了，有請見證人***正清則***
點擊的判斷目前是用拇指、食指、中指的指尖連線夾角cos值判斷，小於-0.7就判定點擊
另外我搞了一個新文件叫[**click_determine**](https://hackmd.io/m7Y3tO3KRUyb2GfARoBHWA?both#click_determine)，用來放點擊判斷的相關自訂義函式，老兄放在後面
現在的問題是：
1. 用cosθ<-0.7 (in click_determine, line28) 似乎不是一個精準的判斷，有時按不下去
2. 在點擊時，鼠標會因為手部動作而跟著移動，導致無法精準點擊


### main
:::spoiler

```py=
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from math import *
from click_determine import distance_counting, click, count_cos

Cam = cv2.VideoCapture(1)                               # 取用相機

# print(ag.position())
# print(ag.size())
Width, Height = ag.size()                               # 取得螢幕尺寸
print(Width, Height)
mp_hands = mp.solutions.hands                           # 手掌偵測方法
mp_drawing = mp.solutions.drawing_utils                 # 繪圖方法
mp_drawing_styles = mp.solutions.drawing_styles         # 繪圖樣式

# 啟用手掌偵測
with mp_hands.Hands(
        model_complexity=0,
        min_tracking_confidence=0.5,
        min_detection_confidence=0.5) as hands:

    while True:
        rec, img = Cam.read()

        if not rec:                                     # 如果打不開鏡頭
            print('BRUH')
            break

        img = cv2.resize(img, (800,600))
        img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)     # CV2讀入的顏色tuple是BGR，要轉換成RGB
        results = hands.process(img2)                   # 識別手部

        # 辨識右手(滑鼠限右手用)(左撇子不友善:((
        if results.multi_handedness:
            # 將手部辨識的結果(左右手)讀入
            for idx, hand_handedness in enumerate(results.multi_handedness):
                # print(hand_handedness.classification)
                # print(hand_handedness.classification[0].label)
                WhichHand = hand_handedness.classification[0].label

            if WhichHand == 'Left':                     # 這邊寫left是因為我的鏡頭左右翻轉
                if results.multi_hand_landmarks:
                    # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                    for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                        destinationX = (1-Hand_Landmark.landmark[9].x)*Width    # 9是中指指根
                        destinationY = (1-Hand_Landmark.landmark[9].y)*Height
                        index_fingerX = Hand_Landmark.landmark[8].x             # 8是食指指尖
                        index_fingerY = Hand_Landmark.landmark[8].y
                        thumb_tipX = Hand_Landmark.landmark[4].x                # 4是大拇指指尖
                        thumb_tipY = Hand_Landmark.landmark[4].y
                        middle_fingerX = Hand_Landmark.landmark[12].x           # 12是中指指尖
                        middle_fingerY = Hand_Landmark.landmark[12].y
                        ring_fingerX = Hand_Landmark.landmark[16].x             # 16是無名指指尖
                        ring_fingerY = Hand_Landmark.landmark[16].y
                        pinkyX = Hand_Landmark.landmark[20].x                   # 20是小拇指指尖
                        pinkyY = Hand_Landmark.landmark[20].y
                        
                        # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                        index_finger = [index_fingerX, index_fingerY]
                        thumb = [thumb_tipX, thumb_tipY]
                        ring_finger = [ring_fingerX, ring_fingerY]
                        
                        ''' # 顯示三指尖夾角cos值，試驗用
                        print(count_cos(distance_counting(index_finger, ring_finger),
                                        distance_counting(index_finger, thumb),
                                        distance_counting(ring_finger, thumb)))'''

                    ag.moveTo(destinationX, destinationY)   # 移動鼠標

                    # 進行點擊的判斷
                    if click(index_finger, thumb, ring_finger):
                        ag.leftClick(destinationX, destinationY)

                    # 將手部識別點繪製於拍攝畫面上
                    for hand_landmarks in results.multi_hand_landmarks:
                        mp_drawing.draw_landmarks(img,
                                                  hand_landmarks,
                                                  mp_hands.HAND_CONNECTIONS,
                                                  mp_drawing_styles.get_default_hand_landmarks_style(),
                                                  mp_drawing_styles.get_default_hand_connections_style())

        cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)       # 顯示拍攝畫面
        if cv2.waitKey(5) == ord('q'):
            break
Cam.release()
cv2.destroyAllWindows
```
:::

### click_determine
:::spoiler
```py=
from math import *


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三邊長算cos用的
def count_cos(a, b, c):
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的主要程式
def click(tip1, tip2, tip3):
    '''
    :param tip1: location of index finger
    :param tip2: location of thumb
    :param tip3: location of ring finger
    :return:
    '''

    a = distance_counting(tip1, tip2)
    b = distance_counting(tip1, tip3)
    c = distance_counting(tip2, tip3)
    cos_theta = count_cos(a, b, c)
    if cos_theta < -0.7:
        return 1

```
:::
### 問題&討論
1. 是不是好像要改成根據手的相對移動來滑動鼠標？還是這樣是我們要的？
2. 以手指動作來判斷點擊的方法要思考一下，有人用手指三節點的角度判斷，我想試試看用五指尖節點相對位置來判斷，說不定還可以增加其他手指的快捷鍵
3. 來不來得及多做一點變化，畢竟這個已經有人做過了

*by ELO*

---
## 6/4 Day3
#### 13:28 
參考了一下別人做的，老兄覺得要改一下程式結構，是時候引進class了(幹
#### 15:15 
老兄剛剛解決了無法移動到邊緣的問題，把鏡頭畫面調大就好了(in main, line30)
不對，根本沒幫助，應該是要調整靈敏度
可以，我成功了
假如想要在座標(a,b)(a<b)之間就可以移動到整個螢幕，也就是(a,b)->(1,0)，可以將讀入的座標套入以下公式
:::success
**x'=b/(b-a)-(x/(b-a))\*width
y'=b/(b-a)-(y/(b-a))\*height**
:::
以程式碼為例，目前我是設定讓移動範圍在座標(0.2,0.7)之間，所以就變成
:::success
**destinationX = ((7/5)-(Hand_Landmark.landmark[9].x/0.5))\*Width 
destinationY = ((7/5)-(Hand_Landmark.landmark[9].y/0.5))\*Height**
:::
新問題
目前的移動模式有個缺點，鼠標是沒辦法固定的，當手部被偵測到時數表就會一直動，沒辦法把鼠標放在一個位置，想想在看影片時首要一直收起來

### main
:::spoiler
```py=
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from math import *
from click_determine import *

Cam = cv2.VideoCapture(1)                               # 取用相機

# print(ag.position())
# print(ag.size())
Width, Height = ag.size()                               # 取得螢幕尺寸
print(Width, Height)
mp_hands = mp.solutions.hands                           # 手掌偵測方法
mp_drawing = mp.solutions.drawing_utils                 # 繪圖方法
mp_drawing_styles = mp.solutions.drawing_styles         # 繪圖樣式
ag.FAILSAFE = False

# 啟用手掌偵測
with mp_hands.Hands(
        model_complexity=1,
        min_tracking_confidence=0.5,
        min_detection_confidence=0.5) as hands:

    while True:
        rec, img = Cam.read()

        if not rec:                                     # 如果打不開鏡頭
            print('BRUH')
            break

        img = cv2.resize(img, (800, 450))
        img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)     # CV2讀入的顏色tuple是BGR，要轉換成RGB
        results = hands.process(img2)                   # 識別手部

        # 辨識右手(滑鼠限右手用)(左撇子不友善:((
        if results.multi_handedness:
            # 將手部辨識的結果(左右手)讀入

            # for idx, hand_handedness in enumerate(results.multi_handedness):
                # WhichHand = hand_handedness.classification[0].label

            if side_detect(results.multi_handedness) == 'Left':                     # 這邊寫left是因為我的鏡頭左右翻轉
                if results.multi_hand_landmarks:
                    # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                    for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                        print(Hand_Landmark.landmark[9])
                        destinationX = ((7/5)-(Hand_Landmark.landmark[9].x/0.5))*Width    # 9是中指指根
                        destinationY = ((7/5)-(Hand_Landmark.landmark[9].y/0.5))*Height     # 若鏡頭對桌面的話，前面要1-才會上下相反
                        '''index_finger_tipX = Hand_Landmark.landmark[8].x             # 8是食指指尖
                        index_finger_tipY = Hand_Landmark.landmark[8].y
                        thumb_tipX = Hand_Landmark.landmark[4].x                # 4是大拇指指尖
                        thumb_tipY = Hand_Landmark.landmark[4].y
                        middle_finger_tipX = Hand_Landmark.landmark[12].x           # 12是中指指尖
                        middle_finger_tipY = Hand_Landmark.landmark[12].y
                        ring_finger_tipX = Hand_Landmark.landmark[16].x             # 16是無名指指尖
                        ring_finger_tipY = Hand_Landmark.landmark[16].y
                        pinky_tipX = Hand_Landmark.landmark[20].x                   # 20是小拇指指尖
                        pinky_tipY = Hand_Landmark.landmark[20].y'''

                        # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                        index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                        thumb = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]
                        ring_finger = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]

                        ''' # 顯示三指尖夾角cos值，試驗用
                        print(count_cos(distance_counting(index_finger, ring_finger),
                                        distance_counting(index_finger, thumb),
                                        distance_counting(ring_finger, thumb)))'''


                    ag.moveTo(destinationX, destinationY)   # 移動鼠標

                    # 進行點擊的判斷
                    # if click(index_finger, thumb, ring_finger):
                        # ag.leftClick(destinationX, destinationY)

                    # 將手部識別點繪製於拍攝畫面上
                    for hand_landmarks in results.multi_hand_landmarks:
                        mp_drawing.draw_landmarks(img,
                                                  hand_landmarks,
                                                  mp_hands.HAND_CONNECTIONS,
                                                  mp_drawing_styles.get_default_hand_landmarks_style(),
                                                  mp_drawing_styles.get_default_hand_connections_style())

        cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)       # 顯示拍攝畫面
        if cv2.waitKey(5) == ord('q'):
            break
Cam.release()
cv2.destroyAllWindows

```
:::

### click_determine
:::spoiler
```py=
from math import *


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三邊長算cos用的
def count_cos(a, b, c):
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的主要程式
def click(tip1, tip2, tip3):
    '''
    :param tip1: location of index finger
    :param tip2: location of thumb
    :param tip3: location of ring finger
    :return:
    '''

    a = distance_counting(tip1, tip2)
    b = distance_counting(tip1, tip3)
    c = distance_counting(tip2, tip3)
    cos_theta = count_cos(a, b, c)
    if cos_theta < -0.7:
        return 1


# 判斷左右手(限右手操控)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label

```
:::
老兄沒有鏡頭可以試試webcam
*by ELO*

---

## 6/5 Day4
老兄今天小累，改一點結構就好
不行，我剛剛有個想法，如果用大拇指當左鍵觸發呢？大拇指的角度看起來頗適合
#### 21:56
<font color ="#FF1212">**不得了了各位，老兄剛剛發現大拇指超適合**</font>
拇指點擊比食指的判斷精準多一百倍，而且不太會有重複觸發的問題，甚至不會影響到鼠標的位置
不過我認為還是需要把兩個子執行緒想辦法分開，像[文獻2](https://blog.csdn.net/Already8888/article/details/126125467)一樣，可以在點擊暫停移動會更好

### main
:::spoiler
```py=
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from math import *
from click_determine import *
import threading
import time

Cam = cv2.VideoCapture(1)                               # 取用相機

# print(ag.position())
# print(ag.size())
Width, Height = ag.size()                               # 取得螢幕尺寸
print(Width, Height)
mp_hands = mp.solutions.hands                           # 手掌偵測方法
mp_drawing = mp.solutions.drawing_utils                 # 繪圖方法
mp_drawing_styles = mp.solutions.drawing_styles         # 繪圖樣式
ag.FAILSAFE = False

# 啟用手掌偵測
with mp_hands.Hands(
        model_complexity=1,
        min_tracking_confidence=0.5,
        min_detection_confidence=0.5) as hands:

    while True:
        rec, img = Cam.read()

        if not rec:                                     # 如果打不開鏡頭
            print('BRUH')
            break

        img = cv2.resize(img, (800, 450))
        img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)     # CV2讀入的顏色tuple是BGR，要轉換成RGB
        results = hands.process(img2)                   # 識別手部

        # 辨識右手(滑鼠限右手用)(左撇子不友善:((
        if results.multi_handedness:
            # 將手部辨識的結果(左右手)讀入

            # for idx, hand_handedness in enumerate(results.multi_handedness):
                # WhichHand = hand_handedness.classification[0].label

            if side_detect(results.multi_handedness) == 'Left':                     # 這邊寫left是因為我的鏡頭左右翻轉
                if results.multi_hand_landmarks:
                    # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                    for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                        # print(Hand_Landmark.landmark[9])
                        destinationX = ((7/5)-(Hand_Landmark.landmark[9].x/0.5))*Width    # 9是中指指根
                        destinationY = ((7/5)-(Hand_Landmark.landmark[9].y/0.5))*Height   # 若鏡頭對桌面的話，前面要1-才會上下相反
                        '''index_finger_tipX = Hand_Landmark.landmark[8].x             # 8是食指指尖
                        index_finger_tipY = Hand_Landmark.landmark[8].y
                        thumb_tipX = Hand_Landmark.landmark[4].x                # 4是大拇指指尖
                        thumb_tipY = Hand_Landmark.landmark[4].y
                        middle_finger_tipX = Hand_Landmark.landmark[12].x           # 12是中指指尖
                        middle_finger_tipY = Hand_Landmark.landmark[12].y
                        ring_finger_tipX = Hand_Landmark.landmark[16].x             # 16是無名指指尖
                        ring_finger_tipY = Hand_Landmark.landmark[16].y
                        pinky_tipX = Hand_Landmark.landmark[20].x                   # 20是小拇指指尖
                        pinky_tipY = Hand_Landmark.landmark[20].y'''

                        # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                        index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                        thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]
                        ring_finger = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]

                        # 拇指指節，拇指指根，嘗試判斷點擊
                        thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                        thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                        '''# 顯示三指尖夾角cos值，試驗用
                        print(count_cos(distance_counting(thumb_ip, thumb_mcp),
                                        distance_counting(thumb_ip, thumb_tip),
                                        distance_counting(thumb_mcp, thumb_tip)))'''

                    ag.moveTo(destinationX, destinationY)   # 移動鼠標

                    # 進行點擊的判斷
                    # if click(index_finger, thumb_tip, ring_finger):
                        # ag.leftClick(destinationX, destinationY)

                    # 點擊判斷2.0-拇指點擊版本(so far so good)
                    if click(thumb_ip, thumb_tip, thumb_mcp):
                        ag.leftClick(destinationX, destinationY)

                    # 將手部識別點繪製於拍攝畫面上
                    for hand_landmarks in results.multi_hand_landmarks:
                        mp_drawing.draw_landmarks(img,
                                                  hand_landmarks,
                                                  mp_hands.HAND_CONNECTIONS,
                                                  mp_drawing_styles.get_default_hand_landmarks_style(),
                                                  mp_drawing_styles.get_default_hand_connections_style())

        cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)       # 顯示拍攝畫面
        if cv2.waitKey(5) == ord('q'):
            break
Cam.release()
cv2.destroyAllWindows

```
:::

### click_determine
:::spoiler
```py=
from math import *


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三邊長算cos用的
def count_cos(a, b, c):
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的主要程式
def click(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return:
    """

    a = distance_counting(point1, point2)
    b = distance_counting(point1, point3)
    c = distance_counting(point2, point3)
    cos_theta = count_cos(a, b, c)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label

```
:::
好，有進度就好
*by ELO*

---

## 6/6 Day5
今天我真的沒做事，老哥幫我們加入了靈敏度的設定，我不確定手指會不會成功保留在偵測範圍內，總之明天試
躺平
喔對了，老兄還有想到，如果改成點擊時暫停移動的話，會無法拖曳，所以這似乎不是個好方法，或是再尋求其他方式達成拖曳功能
*by ELO*

---
## 6/7 Day6
老兄剛剛想到昨天吃飯的時候，跟老哥提到把滑鼠設計成像xbox體感遊戲的操控，就是鏡頭直接水平拍，但是老哥說已經懶得帶滑鼠的人應該不太樂意用電腦還要手舉著
嗯，合理

---
## 6/9 Day7
#### 10:29
昨天沒做事，今天上課先來搞一下
關於鼠標很抖，嘗試改變鏡頭捕捉幀數來精準化手部偵測的位置
鄭清澤提出的sensitivity無法解決我們需要解決的問題，簡單來說，老兄搞錯重點
但是鄭清則同志替我們找到了autogui中的PAUSE，解決了移動次數不夠密集，也就是鼠標的移動不夠絲滑的問題，感謝您，同志
今天修改的部分有：
1. 鼠標移動卡頓問題（ag.PAUSE)
2. 移動到頂端時手指看得到了
3. 增加移動閾值(threshold)來避免手在不移動的情況下亂晃的問題
4. 主程式改為定義事件，main只負責callout

今天的待解問題：
1. 雖然鼠標不移動時可固定，但移動時仍會晃動
2. 滾輪、右鍵、側鍵功能（特別是滾輪，屌打軌跡球關鍵）
3. 左鍵回到食指操控

勁哥的點子：滾輪可以用類似觸控板的雙指驅動

### main
:::spoiler
```py=
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from math import *
from definitions import *
import threading
import time

if __name__ == '__main__':
    hand_detect()
```
:::

### definitions
:::spoiler
```py=
from math import *
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
import threading
import time

# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三邊長算cos用的
def count_cos(a, b, c):
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的主要程式
def click(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return:
    """

    a = distance_counting(point1, point2)
    b = distance_counting(point1, point3)
    c = distance_counting(point2, point3)
    cos_theta = count_cos(a, b, c)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 主程式
def hand_detect():
    Cam = cv2.VideoCapture(0)  # 取用相機
    Cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    # print(ag.position())
    # print(ag.size())
    Width, Height = ag.size()  # 取得螢幕尺寸
    print(Width, Height)
    mp_hands = mp.solutions.hands  # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils  # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles  # 繪圖樣式
    ag.FAILSAFE = False
    prev_position = None
    move_distance = 0
    threshold = 7

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = Cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入

                # for idx, hand_handedness in enumerate(results.multi_handedness):
                # WhichHand = hand_handedness.classification[0].label

                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark.landmark[9])
                            destinationX = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * Width  # 9是中指指根
                            destinationY = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * Height  # 若鏡頭對桌面的話，前面要1-才會上下相反

                            # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                            index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]
                            ring_finger = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]

                            # 拇指指節，拇指指根，嘗試判斷點擊
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            '''# 顯示三指尖夾角cos值，試驗用
                            print(count_cos(distance_counting(thumb_ip, thumb_mcp),
                                            distance_counting(thumb_ip, thumb_tip),
                                            distance_counting(thumb_mcp, thumb_tip)))'''

                        if prev_position is not None:
                            move_distance = distance_counting(prev_position, (destinationX, destinationY))

                        if move_distance > threshold:
                            ag.moveTo(destinationX, destinationY)  # 移動鼠標

                        prev_position = (destinationX, destinationY)

                        # 進行點擊的判斷
                        # if click(index_finger, thumb_tip, ring_finger):
                        # ag.leftClick(destinationX, destinationY)

                        # 點擊判斷2.0-拇指點擊版本(so far so good)
                        if click(thumb_ip, thumb_tip, thumb_mcp):
                            ag.leftClick(destinationX, destinationY)

                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    Cam.release()
    cv2.destroyAllWindows
```
:::

---
## 6/10 Day8
#### 13:02
關於滾輪的部分老兄已經大概有想法了，鏡頭上中指的彎曲相對算明顯，所以我想嘗試直接判斷
老兄順便改了count_cos的內容
### 修改的defs
:::spoiler
```py=
from math import *


def wheel_determine(mid_point, side1_point, side2_point):
    theta = count_cos(mid_point, side1_point, side2_point)
    if theta > threshold: # 等滾輪測試
        return 1
        

def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)
```
:::

---
## 6/11 Day9
#### 19:02
老兄今天凌晨看球，現在超累，不過老兄弄出滾輪了(in definitions, line139-153)
滾輪主要是中指指尖及第一二指節的角度判斷，平時夾角cos值約落在0.98(以我的鏡頭設置而言)
:::success
1. cosθ>-0.75  → 向下捲動(手指彎曲，夾角縮小)
2. cosθ<-0.995 → 向上捲動(手指伸直，角度趨近平角)
:::

目前每次偵測到的移動值是10(像素?不確定)，只要手指持續伸直或彎曲即可向上向下捲動
水啦，好東西
#### 今日問題：
1. 能用是能用，但判定依據也要再調整一下，不然手放著就開始飄，而且還要一直維持特定角度
2. 沒了，目前就這樣

或許可以改成角度改變才滾動，而不是到特定角度就滾動，但是再說，我要去吃飯
喔或是改用一二指節和指根的角度

### main
:::spoiler
```py=
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from math import *
from definitions import *
import threading
import time


if __name__ == '__main__':
    hand_detect()

```
:::

### definitions
:::spoiler
```py=
from math import *
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from definitions import *
import threading
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三座標算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的程式
def click(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return:
    """

    cos_theta = count_cos(point1, point2, point3)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


# 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(1)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    # print(ag.position())
    # print(ag.size())
    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands  # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils  # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles  # 繪圖樣式
    ag.FAILSAFE = False
    prev_position = None
    move_distance = 0
    threshold = 9

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark.landmark[9])
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height  # 若鏡頭對桌面的話，前面要1-才會上下相反

                            # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                            index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]

                            # 拇指指節、拇指指根，嘗試判斷點擊
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_dip = [Hand_Landmark.landmark[11].x, Hand_Landmark.landmark[11].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                            # 顯示三指尖夾角cos值，試驗用
                            # print(count_cos(mid_dip, mid_pip, mid_tip))
                                
                            
                            # 防靜止晃動的移動量判斷
                            if prev_position is not None:
                                move_distance = distance_counting(prev_position, (destination_x, destination_y))

                            if move_distance > threshold:
                                ag.moveTo(destination_x, destination_y)       # 移動鼠標
                                
                            prev_position = (destination_x, destination_y)    # 紀錄現在位置以供下次迴圈判斷移動量

                            # 進行點擊的判斷
                            # if click(index_finger, thumb_tip, mid_finger):
                            # ag.leftClick(destinationX, destinationY)

                            # 點擊判斷2.0-拇指點擊版本(so far so good)
                            if click(thumb_ip, thumb_tip, thumb_mcp):
                                ag.leftClick(destination_x, destination_y)

                            # 偵測滾輪的捲動
                            wheel_scroll(mid_dip, mid_tip, mid_pip)

                            # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.75:           # 判斷為彎曲
        return 1
    elif theta < -0.9965:       # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-10)      # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(10)       # 向上捲動

```
:::

---
## 6/14 Day 10
#### 08:26
廢了三天，老兄要沒料了
目前好像可以弄一下重複點擊的問題，今天下午自主學習可以試試看，還有開始用右鍵好了
幹，還有食指改斜率判斷
接下來在defs中要加入tan的運算和判斷，試試看能不能解決點擊判定不佳的問題
先睡ㄌ

---
## 6/16 Day11
#### 00:45
老兄剛剛想到，<font color='961573'>要用一個設定可以關掉滑鼠功能</font>，也就是在開著程式時可以離開滑鼠而不會打字時一直動到游標
#### 10:33
今天的待處理事項：
1. 小指右鍵
2. 手勢關滑鼠

修改了中指的判定，從指尖、一二指節改成指尖、二指節和指根

### Roger_Try_count_angle(but failed)
:::spoiler
```py=
def finger_angle(v1, v2):
    v1_x = v1[0]
    v1_y = v1[1]
    v2_x = v2[0]
    v2_y = v2[1]
    try:
        angle_ = degrees(acos((v1_x*v2_x+v1_y*v2_y)/(((v1_x**2+v1_y**2)**0.5)*((v2_x**2+v2_y**2)**0.5))))
    except:
        angle_ = 180
    return angle_


def hand_angle(hand_):
    angle_list = []
    angle_ = finger_angle(
        ((int(hand_[0][0]) - int(hand_[18][0])), (int(hand_[0][1]) - int(hand_[18][1]))),
        ((int(hand_[19][0]) - int(hand_[20][0])), (int(hand_[19][1]) - int(hand_[20][1])))
    )


def right_click(finger_angle):
    small_finger = finger_angle[0]

    if small_finger<50:
        return 'good'
    elif small_finger>=50:
        return 'nice'
```
:::

### main
:::spoiler
```py=

```
:::

### definitions
:::spoiler
```py=
from math import *
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from definitions import *
import threading
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三座標算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 左鍵點擊判斷的程式
def click_left(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return: whether to click
    """

    cos_theta = count_cos(point1, point2, point3)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


# 右鍵點擊
def click_right(p1,p2,p3):
    cos_theta = count_cos(p1, p2, p3)
    if cos_theta > -0.95:  # 這是拇指判斷的值
        return 1


    # 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(0)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    # print(ag.position())
    # print(ag.size())
    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands  # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils  # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles  # 繪圖樣式
    ag.FAILSAFE = False
    prev_position = None
    move_distance = 0
    threshold = 9

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark.landmark[9])
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height  # 若鏡頭對桌面的話，前面要1-才會上下相反

                            # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                            index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]

                            # 拇指指節、拇指指根，嘗試判斷點擊
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_dip = [Hand_Landmark.landmark[11].x, Hand_Landmark.landmark[11].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                            pinky_tip =[Hand_Landmark.landmark[20].x, Hand_Landmark.landmark[20].y]
                            pinky_dip =[Hand_Landmark.landmark[19].x, Hand_Landmark.landmark[19].y]
                            pinky_pip =[Hand_Landmark.landmark[18].x, Hand_Landmark.landmark[18].y]
                            pinky_mcp =[Hand_Landmark.landmark[17].x, Hand_Landmark.landmark[17].y]

                            # 顯示三指尖夾角cos值，試驗用
                            print(count_cos(pinky_dip, pinky_tip, pinky_pip))

                        # 防靜止晃動的移動量判斷
                        if prev_position is not None:
                            move_distance = distance_counting(prev_position, (destination_x, destination_y))

                        if move_distance > threshold:
                            ag.moveTo(destination_x, destination_y)       # 移動鼠標

                        prev_position = (destination_x, destination_y)    # 紀錄現在位置以供下次迴圈判斷移動量

                        # 進行點擊的判斷
                        # if click(index_finger, thumb_tip, mid_finger):
                        # ag.leftClick(destinationX, destinationY)

                        # 點擊判斷2.0-拇指點擊版本(so far so good)
                        if click_left(thumb_ip, thumb_tip, thumb_mcp):
                            ag.leftClick(destination_x, destination_y)

                        if click_right(pinky_dip, pinky_tip, pinky_pip):
                            ag.rightClick(destination_x, destination_y)

                        # 偵測滾輪的捲動
                        #wheel_scroll(mid_pip, mid_tip, mid_mcp)



                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.76:           # 判斷為彎曲
        return 1
    elif theta < -0.9965:       # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-10)      # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(10)       # 向上捲動






```
:::


>勁哥程式貼這邊👇
>[color=#a162cb]
:::spoiler
```py=
from math import *
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from definitions import *
import threading
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三座標算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的程式
def click(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return:
    """

    cos_theta = count_cos(point1, point2, point3)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


# 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(0)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    # print(ag.position())
    # print(ag.size())
    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands  # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils  # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles  # 繪圖樣式
    ag.FAILSAFE = False
    prev_position = None
    move_distance = 0
    threshold = 9

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark.landmark[9])
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height  # 若鏡頭對桌面的話，前面要1-才會上下相反

                            # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                            index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]

                            # 拇指指節、拇指指根，嘗試判斷點擊
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_dip = [Hand_Landmark.landmark[11].x, Hand_Landmark.landmark[11].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                            landmarks = Hand_Landmark.landmark
                            # 顯示三指尖夾角cos值，試驗用
                            # print(count_cos(mid_dip, mid_pip, mid_tip))

                        # 防靜止晃動的移動量判斷
                        if prev_position is not None:
                            move_distance = distance_counting(prev_position, (destination_x, destination_y))
                        switcher = 0
                        if move_distance > threshold:
                            switcher = gesture_identify(landmarks)
                            if switcher == 1:
                                cv2.putText(img, 'close', (30, 120),  cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255))
                                switcher_pause = 0
                                while  switcher_pause != 2:
                                    results = hands.process(img2)  # 識別手部
                                    # 辨識右手(滑鼠限右手用)(左撇子不友善:((
                                    if results.multi_handedness:
                                        # 將手部辨識的結果(左右手)讀入
                                        if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                                            if results.multi_hand_landmarks:
                                                # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                                                for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                                                    # print(Hand_Landmark.landmark[9])
                                                    destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                                                    destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height  # 若鏡頭對桌面的話，前面要1-才會上下相反
                                                    switcher_pause = gesture_identify(Hand_Landmark.landmark)
                                ag.moveTo(destination_x, destination_y)
                                cv2.putText(img, 'open', (30, 120), cv2.FONT_HERSHEY_SIMPLEX, 1, (255, 255, 255))
                            else:
                                ag.moveTo(destination_x, destination_y)       # 移動鼠標

                        prev_position = (destination_x, destination_y)    # 紀錄現在位置以供下次迴圈判斷移動量

                        # 進行點擊的判斷
                        # if click(index_finger, thumb_tip, mid_finger):
                        # ag.leftClick(destinationX, destinationY)

                        # 點擊判斷2.0-拇指點擊版本(so far so good)
                        if click(thumb_ip, thumb_tip, thumb_mcp):
                            ag.leftClick(destination_x, destination_y)

                        # 偵測滾輪的捲動
                        wheel_scroll(mid_dip, mid_tip, mid_pip)

                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.75:           # 判斷為彎曲
        return 1
    elif theta < -0.9965:       # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-10)      # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(10)       # 向上捲動

def gesture_identify(marks):
    thumb_mcp = [marks[2].x, marks[2].y]
    thumb_ip = [marks[3].x, marks[3].y]
    thumb_tip = [marks[4].x, marks[4].y]
    ind_pip = [marks[6].x, marks[6].y]
    ind_dip = [marks[7].x, marks[7].y]
    ind_tip = [marks[8].x, marks[8].y]
    mid_pip = [marks[10].x, marks[10].y]
    mid_dip = [marks[11].x, marks[11].y]
    mid_tip = [marks[12].x, marks[12].y]
    ring_pip = [marks[14].x, marks[14].y]
    ring_dip = [marks[15].x, marks[15].y]
    ring_tip = [marks[16].x, marks[16].y]
    pinky_pip = [marks[18].x, marks[18].y]
    pinky_dip = [marks[19].x, marks[19].y]
    pinky_tip = [marks[20].x, marks[20].y]

    thu = wheel_determine(thumb_ip,thumb_tip, thumb_mcp )
    ind = wheel_determine(ind_dip, ind_tip, ind_pip)
    mid = wheel_determine(mid_dip, mid_tip, mid_pip)
    rin = wheel_determine(ring_dip, ring_tip, ring_pip)
    pin = wheel_determine(pinky_dip, pinky_tip, pinky_pip)

    if thu == 1 and ind == 1 and mid == 2 and rin == 2 and pin == 2:
        return 1
    elif thu == 1 and ind == 2 and mid == 2 and rin == 2 and pin == 2:
        return 2
    return 0

```
:::
基本上這個開關弄完就可以demo了

---

## 6/17 Day12
老哥們今天加油，主要完成手勢開關，記得註記一下是什麼手勢，如果還有時間的話，可以把程式直接改成可開可關
關於開關的方式，我這邊有一個想法：
```py=
def gesture_identify(thumb, index, mid, ring, pinky, counter # 用以計算開關的次數):
    # 這邊放判斷手勢的程式
    if OK:    #假如用ＯＫ
        counter + = 1
    if counter % 2 == 0: 
        return 1
```
counter負責計算開關次數，若第一次開則會將之變1，並因為是奇數而關閉滑鼠，融入主程式要注意，迴圈要先偵測開關才能進入滑鼠，不然在迴圈內偵測開關會導致關掉之後開不起來

#### 11:17
由於判斷是連續數幀的畫面，根據上面的想法判斷後會發現計數器將重複被記次，導致無法順利開關。改以跳至新迴圈來避免重複觸發的問題，當偵測到手勢，將跳至一個空的無限迴圈，直到無限迴圈再度偵測到手勢才會跳回原程式。
#### 18:50
從勁哥貼的那邊看來，手指彎曲是用wheel_determine檢測，但每隻手指從鏡頭下看來的角度其實不一定一樣，或許每隻手指到彎曲判定再print測試一下後改theta判斷的值就會好一點？（就是每個都獨立有一個角度判斷的事件，因為他們的角度都不一樣）
還有如果要以三點判斷角度的話，可能tip, pip, mcp的夾角會比較明顯。

---

## 6/18 Day13
#### 00:33
明天修手勢的問題，如果手勢修好了但是迴圈上還是不能跑的話，就要引進threading了，threading中有一個功能叫做鎖定(lock)，如果兩個平行執行緒共用同一個鎖定，那麼就只會執行第一個鎖定的執行緒，其他則要等其解鎖後才執行
我們可以利用這個特性，將主程式設為第一個鎖定的執行緒，偵測到手勢後就將其解鎖(lock.release)，則第二執行緒(也就是暫停偵測的部分)執行，在第二執行緒中再度偵測手勢，偵測到後將執行緒一上鎖(lock.acquire)，預計就會再度執行主程式。

<font color='b134ac'>**e04 沒用啊**</font>
早上我再想想，我原本以為上鎖可以這樣用但是好像不行，當解鎖後就只會兩者一起繼續執行，而不是停下第一執行緒
早上再看一下threading.Event，我需要睡眠
![](https://hackmd.io/_uploads/rJLCCDiw3.png)

輕則思考版本
![](https://hackmd.io/_uploads/Hk6ddyCD3.png)


#### 12:16
其實根本不用那麼麻煩用threading，迴圈可以解決(半夜到底在幹嘛)
概念如下：
```py=
import time

# 原本不論輸入甚麼，都會移動(輸出moving)
# 如果輸入ok(也就是拍到ok)，則停止移動
# 接著輸入其他手勢將會not moving
# 再次輸入ok，也就是再次偵測到ok，則回到移動(moving)


def ok(n):
    if n == 'ok':
        print('OK')
        return 1
    else:
        return 0


def move():
    print('moving')
    time.sleep(0.5)


identifier = 0
while True:
    gesture = input('輸入手勢：')
    if not identifier:
        if ok(gesture):
            identifier = 1
        else:
            move()
    else:
        print('not moving')
        if ok(gesture):
            identifier = 0


```
這段是以輸入文字代替攝影機拍照，當偵測到ok也就是手勢為ok，一開始輸入任何手勢都會輸出moving，輸入第一次ok後，輸出OK表示有偵測到ok手勢，再次輸入任何手勢則會輸出not moving。
輸入第二次ok後，才會再次回到moving，大概是這個概念。
```mermaid
graph LR;
拍攝手部-- ok-->再次拍攝手部;
拍攝手部-- not ok-->移動;
移動-->拍攝手部;
再次拍攝手部--not ok-->再次拍攝手部;
再次拍攝手部-- ok-->拍攝手部;
```
我這邊改成每次都回到大迴圈，以if, else進行分流，這樣可以確保每次都會拍攝到手部(輸入手勢)，跟勁哥原本的概念不太一樣，上面的程式看起來是偵測到ok後進入一個小迴圈偵測手勢，但小回圈內我沒有看到拍攝的動作，而且再度拍攝就要再寫一次獨入座標的動作，可能程式會有重複，因此如此改動。

### def最新版 6/18
:::spoiler
```py=
from math import *
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from definitions import *
import threading
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三邊長算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的主要程式
def click(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return:
    """

    cos_theta = count_cos(point1, point2, point3)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


# 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(1)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    # print(ag.position())
    # print(ag.size())
    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands  # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils  # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles  # 繪圖樣式
    ag.FAILSAFE = False
    prev_position = None
    move_distance = 0
    threshold = 9

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark)
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height  # 若鏡頭對桌面的話，前面要1-才會上下相反

                            # 食指，中指，大拇指指尖座標的list(點擊判斷用)
                            index_finger = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]

                            # 拇指指節、拇指指根，嘗試判斷點擊
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 食指指尖、指節1、指節2、指根
                            index_tip = [Hand_Landmark.landmark[8].x, Hand_Landmark.landmark[8].y]
                            index_dip = [Hand_Landmark.landmark[7].x, Hand_Landmark.landmark[7].y]
                            index_pip = [Hand_Landmark.landmark[6].x, Hand_Landmark.landmark[6].y]
                            index_mcp = [Hand_Landmark.landmark[5].x, Hand_Landmark.landmark[5].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_dip = [Hand_Landmark.landmark[11].x, Hand_Landmark.landmark[11].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                            # 無名指指尖、指節1、指節2、指根
                            ring_tip = [Hand_Landmark.landmark[16].x, Hand_Landmark.landmark[16].y]
                            ring_dip = [Hand_Landmark.landmark[15].x, Hand_Landmark.landmark[15].y]
                            ring_pip = [Hand_Landmark.landmark[14].x, Hand_Landmark.landmark[14].y]
                            ring_mcp = [Hand_Landmark.landmark[13].x, Hand_Landmark.landmark[13].y]

                            # 小指指尖、第一、二指節及指根
                            pinky_mcp =[Hand_Landmark.landmark[17].x, Hand_Landmark.landmark[17].y]
                            pinky_pip =[Hand_Landmark.landmark[18].x, Hand_Landmark.landmark[18].y]
                            pinky_dip =[Hand_Landmark.landmark[19].x, Hand_Landmark.landmark[19].y]
                            pinky_tip =[Hand_Landmark.landmark[20].x, Hand_Landmark.landmark[20].y]

                            # 顯示三指尖夾角cos值，試驗用
                            # print('=============================')
                            # print(count_cos(thumb_ip, thumb_mcp, thumb_tip))
                            # print(count_cos(index_pip, index_mcp, mid_tip))
                            # print(count_cos(mid_pip, mid_mcp, mid_tip))
                            # print(count_cos(ring_pip, ring_mcp, ring_tip))
                            # print(count_cos(pinky_pip, pinky_mcp, pinky_tip))

                        if not identifier:
                            if gesture_determine(Hand_Landmark.landmark):
                                identifier = 1
                                
                            else:
                            # 防靜止晃動的移動量判斷
                            if prev_position is not None:
                                move_distance = distance_counting(prev_position, (destination_x, destination_y))

                            if move_distance > threshold:
                                ag.moveTo(destination_x, destination_y)       # 移動鼠標
    
                            prev_position = (destination_x, destination_y)    # 紀錄現在位置以供下次迴圈判斷移動量
        
                            # 進行點擊的判斷
                            # if click(index_finger, thumb_tip, mid_finger):
                            # ag.leftClick(destinationX, destinationY)
    
                            # 點擊判斷2.0-拇指點擊版本(so far so good)
                            if click(thumb_ip, thumb_tip, thumb_mcp):
                                ag.leftClick(destination_x, destination_y)

                            # 偵測滾輪的捲動
                            wheel_scroll(mid_dip, mid_tip, mid_pip)
                            
                        else:
                            if gesture_determine(Hand_Landmark.landmark):
                                identifier = 0

                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.75:           # 判斷為彎曲
        return 1
    elif theta < -0.9965:       # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-10)      # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(10)       # 向上捲動


'''def count_tangent(point1, point2):
    x1, y1, x2, y2 = point1[0], point1[1], point2[0], point2[1]
    tangent = '''


def gesture_identify(marks):
    thumb_mcp = [marks[2].x, marks[2].y]
    thumb_ip = [marks[3].x, marks[3].y]
    thumb_tip = [marks[4].x, marks[4].y]
    ind_pip = [marks[6].x, marks[6].y]
    ind_dip = [marks[7].x, marks[7].y]
    ind_tip = [marks[8].x, marks[8].y]
    mid_pip = [marks[10].x, marks[10].y]
    mid_dip = [marks[11].x, marks[11].y]
    mid_tip = [marks[12].x, marks[12].y]
    ring_pip = [marks[14].x, marks[14].y]
    ring_dip = [marks[15].x, marks[15].y]
    ring_tip = [marks[16].x, marks[16].y]
    pinky_pip = [marks[18].x, marks[18].y]
    pinky_dip = [marks[19].x, marks[19].y]
    pinky_tip = [marks[20].x, marks[20].y]

    thu = wheel_determine(thumb_ip,thumb_tip, thumb_mcp )
    ind = wheel_determine(ind_dip, ind_tip, ind_pip)
    mid = wheel_determine(mid_dip, mid_tip, mid_pip)
    rin = wheel_determine(ring_dip, ring_tip, ring_pip)
    pin = wheel_determine(pinky_dip, pinky_tip, pinky_pip)

    if thu == 1 and ind == 1 and mid == 2 and rin == 2 and pin == 2:
        return 1
    elif thu == 1 and ind == 2 and mid == 2 and rin == 2 and pin == 2:
        return 2
    return 0

```
:::
等勁哥的手勢判斷

---

## 6/19 Day14
#### 10:37
剩手指角度判斷，昨天試了一下發現ok的角度變化不是很大，在偵測方面要有不靈敏的心理準備
而迴圈的部分已經處理好了，把上次的帶進去了，等總測試確定能用就來demo
然後老兄們先把報告弄出來，程式解說的部分可以先貼圖片就好，不確定拿來幹嘛的部分先留著我之後補
#### 21:30
弄出來了

### main
:::spoiler
```py=
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
from math import *
from definitions import *
import threading
import time


if __name__ == '__main__':
    hand_detect()

```
:::

### definitions
:::spoiler
```py=
from math import *
import mediapipe as mp          # 手形辨識
import cv2                      # 取用相機
import pyautogui as ag          # 移動滑鼠
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0]-loc2[0])**2+(loc1[1]-loc2[1])**2)
    return distance


# 以三邊長算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2)/(2 * a * b)


# 點擊判斷的主要程式
def click(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return:
    """

    cos_theta = count_cos(point1, point2, point3)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


# 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.75:           # 判斷為彎曲
        return 1
    elif theta < -0.9965:       # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-20)      # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(20)       # 向上捲動


# 開關用手勢判斷(OK)
def gesture_identify(marks):
    # 拇指節點
    thumb_mcp = [marks[2].x, marks[2].y]
    thumb_ip = [marks[3].x, marks[3].y]
    thumb_tip = [marks[4].x, marks[4].y]
    # 食指節點
    ind_mcp = [marks[5].x, marks[5].y]
    ind_pip = [marks[6].x, marks[6].y]
    ind_tip = [marks[8].x, marks[8].y]
    # 中指節點
    mid_mcp = [marks[9].x, marks[9].y]
    mid_pip = [marks[10].x, marks[10].y]
    mid_tip = [marks[12].x, marks[12].y]
    ring_mcp = [marks[13].x, marks[13].y]
    ring_pip = [marks[14].x, marks[14].y]
    ring_tip = [marks[16].x, marks[16].y]
    # 小指節點
    pinky_mcp = [marks[17].x, marks[17].y]
    pinky_pip = [marks[18].x, marks[18].y]
    pinky_tip = [marks[20].x, marks[20].y]

    thu = count_cos(thumb_ip, thumb_tip, thumb_mcp)
    ind = count_cos(ind_pip, ind_tip, ind_mcp)
    mid = count_cos(mid_pip, mid_tip, mid_mcp)
    rin = count_cos(ring_pip, ring_tip, ring_mcp)
    pin = count_cos(pinky_pip, pinky_tip, pinky_mcp)

    # 拇指、食指：彎曲
    # 中指、無名指、小指：伸直
    if thu > -0.94 and ind > -0.9 and mid > -0.988 and rin > -0.998 and pin > -0.998:
        return 1
    else:
        return 0


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(1)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    # print(ag.position())
    # print(ag.size())
    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands  # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils  # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles  # 繪圖樣式
    ag.FAILSAFE = False
    prev_position = None
    move_distance = 0
    threshold = 9
    identifier = 0

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark)
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height

                            # 拇指指尖、指節、拇指指根，嘗試判斷點擊
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                        if gesture_identify(Hand_Landmark.landmark):
                            print('ok')

                        # 手勢開關
                        if not identifier:
                            # 可移動時偵測到OK：關閉移動功能
                            if gesture_identify(Hand_Landmark.landmark):
                                identifier = 1
                                print('pause')
                                time.sleep(1)

                            # 移動&點擊
                            else:
                                # 防靜止晃動的移動量判斷
                                if prev_position is not None:
                                    move_distance = distance_counting(prev_position, (destination_x, destination_y))
        
                                if move_distance > threshold:
                                    ag.moveTo(destination_x, destination_y)       # 移動鼠標

                                # 紀錄現在位置以供下次迴圈判斷移動量
                                prev_position = (destination_x, destination_y)
        
                                # 點擊判斷-拇指點擊版本(so far so good)
                                if click(thumb_ip, thumb_tip, thumb_mcp):
                                    ag.leftClick(destination_x, destination_y)
        
                                # 偵測滾輪的捲動
                                print(count_cos(mid_pip, mid_tip, mid_mcp))
                                wheel_scroll(mid_pip, mid_tip, mid_mcp)
                                
                        else:
                            if gesture_identify(Hand_Landmark.landmark):
                                time.sleep(1)
                                identifier = 0

                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows

```
:::
準備demo囉各位

### FINAL main
最後我把definitions跟main合併了，變這樣：
:::spoiler
```py=
from math import *
import mediapipe as mp  # 手形辨識
import cv2              # 取用相機
import pyautogui as ag  # 移動滑鼠
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0] - loc2[0]) ** 2 + (loc1[1] - loc2[1]) ** 2)
    return distance


# 以三邊長算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2) / (2 * a * b)


# 左鍵點擊判斷的程式
def click_left(point1, point2, point3):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return: whether to click
    """

    cos_theta = count_cos(point1, point2, point3)
    if cos_theta > -0.9:        # 這是拇指判斷的值
        return 1                # 指尖判斷的值是cos_theta<-0.7


# 右鍵點擊
def click_right(p1, p2, p3):
    cos_theta = count_cos(p1, p2, p3)
    if cos_theta > -0.8:  # 這是拇指判斷的值
        return 1


# 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.75:  # 判斷為彎曲
        return 1
    elif theta < -0.9965:  # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-20)  # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(20)  # 向上捲動


# 開關用手勢判斷(OK)
def gesture_identify(marks):
    # 拇指節點
    thumb_mcp = [marks[2].x, marks[2].y]
    thumb_ip = [marks[3].x, marks[3].y]
    thumb_tip = [marks[4].x, marks[4].y]
    # 食指節點
    ind_mcp = [marks[5].x, marks[5].y]
    ind_pip = [marks[6].x, marks[6].y]
    ind_tip = [marks[8].x, marks[8].y]
    # 中指節點
    mid_mcp = [marks[9].x, marks[9].y]
    mid_pip = [marks[10].x, marks[10].y]
    mid_tip = [marks[12].x, marks[12].y]
    ring_mcp = [marks[13].x, marks[13].y]
    ring_pip = [marks[14].x, marks[14].y]
    ring_tip = [marks[16].x, marks[16].y]
    # 小指節點
    pinky_mcp = [marks[17].x, marks[17].y]
    pinky_pip = [marks[18].x, marks[18].y]
    pinky_tip = [marks[20].x, marks[20].y]

    # 計算手指角度cos值
    thu = count_cos(thumb_ip, thumb_tip, thumb_mcp)
    ind = count_cos(ind_pip, ind_tip, ind_mcp)
    mid = count_cos(mid_pip, mid_tip, mid_mcp)
    rin = count_cos(ring_pip, ring_tip, ring_mcp)
    pin = count_cos(pinky_pip, pinky_tip, pinky_mcp)

    # 拇指、食指：彎曲
    # 中指、無名指、小指：伸直
    if thu > -0.94 and ind > -0.9 and mid > -0.988 and rin > -0.998 and pin > -0.998:
        return 1
    else:
        return 0


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(1)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands                       # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils             # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles     # 繪圖樣式
    ag.FAILSAFE = False

    # 為移動閾值準備
    prev_position = None
    move_distance = 0
    threshold = 9

    # 為手勢開關準備
    identifier = 0

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark)
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height

                            # 拇指指尖、指節、拇指指根，嘗試判斷點擊
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                            # 小指指尖、指節、指根
                            pinky_tip = [Hand_Landmark.landmark[20].x, Hand_Landmark.landmark[20].y]
                            pinky_pip = [Hand_Landmark.landmark[18].x, Hand_Landmark.landmark[18].y]
                            pinky_mcp = [Hand_Landmark.landmark[17].x, Hand_Landmark.landmark[17].y]

                        if gesture_identify(Hand_Landmark.landmark):
                            print('ok')

                        # 手勢開關
                        if not identifier:
                            # 可移動時偵測到OK：關閉移動功能
                            if gesture_identify(Hand_Landmark.landmark):
                                identifier = 1
                                print('pause')
                                time.sleep(1)

                            # 移動&點擊
                            else:
                                # 防靜止晃動的移動量判斷
                                if prev_position is not None:
                                    move_distance = distance_counting(prev_position, (destination_x, destination_y))

                                if move_distance > threshold:
                                    ag.moveTo(destination_x, destination_y)  # 移動鼠標

                                # 紀錄現在位置以供下次迴圈判斷移動量
                                prev_position = (destination_x, destination_y)

                                # 點擊判斷-拇指點擊版本(so far so good)
                                if click_left(thumb_ip, thumb_tip, thumb_mcp):
                                    ag.leftClick(destination_x, destination_y)

                                if click_right(pinky_pip, pinky_tip, pinky_mcp):
                                    ag.rightClick(destination_x, destination_y)

                                # 偵測滾輪的捲動
                                wheel_scroll(mid_pip, mid_tip, mid_mcp)

                        else:
                            # 關閉時偵測到OK：開啟移動功能
                            if gesture_identify(Hand_Landmark.landmark):
                                time.sleep(1)
                                identifier = 0

                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows


if __name__ == '__main__':
    hand_detect()


```
:::
終於可以安穩地睡覺了
*by ELO*

---

## 最終成品
```py=
from math import *
import mediapipe as mp  # 手形辨識
import cv2              # 取用相機
import pyautogui as ag  # 移動滑鼠
import time


# 很明顯，算距離用的
def distance_counting(loc1, loc2):
    distance = sqrt((loc1[0] - loc2[0]) ** 2 + (loc1[1] - loc2[1]) ** 2)
    return distance


# 以三邊長算cos用的
def count_cos(mid, side1, side2):
    a = distance_counting(mid, side1)
    b = distance_counting(mid, side2)
    c = distance_counting(side1, side2)
    return (a ** 2 + b ** 2 - c ** 2) / (2 * a * b)


# 左鍵點擊判斷的程式
def click_left(point1, point2, point3, position):
    """
    :param point1: location of index finger
    :param point2: location of thumb
    :param point3: location of ring finger
    :return: whether to click
    """

    cos_theta = count_cos(point1, point2, point3)
    # 根據手部位置不同拍攝出角度會不同
    if position[0] > 0.5:           # 相機右側
        if cos_theta > -0.85:       # 這是拇指判斷的值
            return 1
    else:                           # 相機左側
        if cos_theta > -0.9:        # 這是拇指判斷的值
            return 1


# 右鍵點擊
def click_right(p1, p2, p3, position):
    cos_theta = count_cos(p1, p2, p3)
    # 根據手部位置不同拍攝出角度會不同
    if position[0] > 0.5:       # 相機右側
        if cos_theta > -0.79:   # 這是小拇指判斷的值
            return 1
    else:                       # 相機左側
        if cos_theta > -0.7:    # 這是小拇指判斷的值
            return 1


# 判斷偵測到的手(左右相反)
def side_detect(handedness):
    for idx, hand_handedness in enumerate(handedness):
        return hand_handedness.classification[0].label


# 檢測中指彎曲
def wheel_determine(mid_point, side_point1, side_point2):
    theta = count_cos(mid_point, side_point1, side_point2)
    if theta > -0.75:  # 判斷為彎曲
        return 1
    elif theta < -0.9965:  # 判斷為伸直
        return 2


# 滾輪滾動事件
def wheel_scroll(mid, side1, side2):
    if wheel_determine(mid, side1, side2) == 1:
        ag.scroll(-20)  # 向下捲動
    elif wheel_determine(mid, side1, side2) == 2:
        ag.scroll(20)  # 向上捲動


# 開關用手勢判斷(OK)
def gesture_identify(marks):
    # 拇指節點
    thumb_mcp = [marks[2].x, marks[2].y]
    thumb_ip = [marks[3].x, marks[3].y]
    thumb_tip = [marks[4].x, marks[4].y]
    # 食指節點
    ind_mcp = [marks[5].x, marks[5].y]
    ind_pip = [marks[6].x, marks[6].y]
    ind_tip = [marks[8].x, marks[8].y]
    # 中指節點
    mid_mcp = [marks[9].x, marks[9].y]
    mid_pip = [marks[10].x, marks[10].y]
    mid_tip = [marks[12].x, marks[12].y]
    ring_mcp = [marks[13].x, marks[13].y]
    ring_pip = [marks[14].x, marks[14].y]
    ring_tip = [marks[16].x, marks[16].y]
    # 小指節點
    pinky_mcp = [marks[17].x, marks[17].y]
    pinky_pip = [marks[18].x, marks[18].y]
    pinky_tip = [marks[20].x, marks[20].y]

    # 計算手指角度cos值
    thu = count_cos(thumb_ip, thumb_tip, thumb_mcp)
    ind = count_cos(ind_pip, ind_tip, ind_mcp)
    mid = count_cos(mid_pip, mid_tip, mid_mcp)
    rin = count_cos(ring_pip, ring_tip, ring_mcp)
    pin = count_cos(pinky_pip, pinky_tip, pinky_mcp)

    # 拇指、食指：彎曲
    # 中指、無名指、小指：伸直
    if thu > -0.94 and ind > -0.9 and mid > -0.988 and rin > -0.998 and pin > -0.998:
        return 1
    else:
        return 0


# 主程式
def hand_detect():
    cam = cv2.VideoCapture(1)  # 取用相機
    cam.set(cv2.CAP_PROP_FPS, 120)
    ag.PAUSE = 0.001

    width, height = ag.size()  # 取得螢幕尺寸
    print(width, height)
    mp_hands = mp.solutions.hands                       # 手掌偵測方法
    mp_drawing = mp.solutions.drawing_utils             # 繪圖方法
    mp_drawing_styles = mp.solutions.drawing_styles     # 繪圖樣式
    ag.FAILSAFE = False

    # 為移動閾值準備
    prev_position = None
    move_distance = 0
    threshold = 9

    # 為手勢開關準備
    identifier = 0

    # 啟用手掌偵測
    with mp_hands.Hands(
            model_complexity=1,
            min_tracking_confidence=0.5,
            min_detection_confidence=0.5) as hands:

        while True:
            rec, img = cam.read()

            if not rec:  # 如果打不開鏡頭
                print('BRUH')
                break

            img = cv2.resize(img, (800, 450))
            img2 = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)  # CV2讀入的顏色tuple是BGR，要轉換成RGB
            results = hands.process(img2)  # 識別手部

            # 辨識右手(滑鼠限右手用)(左撇子不友善:((
            if results.multi_handedness:
                # 將手部辨識的結果(左右手)讀入
                if side_detect(results.multi_handedness) == 'Left':  # 這邊寫left是因為我的鏡頭左右翻轉
                    if results.multi_hand_landmarks:
                        # 將手部辨識結果(座標)讀入，並轉成螢幕的座標
                        for ind, Hand_Landmark in enumerate(results.multi_hand_landmarks):
                            # print(Hand_Landmark)
                            destination_x = ((6 / 4) - (Hand_Landmark.landmark[9].x / 0.4)) * width  # 9是中指指根
                            destination_y = ((6 / 4) - (Hand_Landmark.landmark[9].y / 0.4)) * height

                            # 拇指指尖、指節、拇指指根，嘗試判斷點擊
                            thumb_tip = [Hand_Landmark.landmark[4].x, Hand_Landmark.landmark[4].y]
                            thumb_ip = [Hand_Landmark.landmark[3].x, Hand_Landmark.landmark[3].y]
                            thumb_mcp = [Hand_Landmark.landmark[2].x, Hand_Landmark.landmark[2].y]

                            # 中指指尖、指節1、指節2、指根
                            mid_tip = [Hand_Landmark.landmark[12].x, Hand_Landmark.landmark[12].y]
                            mid_pip = [Hand_Landmark.landmark[10].x, Hand_Landmark.landmark[10].y]
                            mid_mcp = [Hand_Landmark.landmark[9].x, Hand_Landmark.landmark[9].y]

                            # 小指指尖、指節、指根
                            pinky_tip = [Hand_Landmark.landmark[20].x, Hand_Landmark.landmark[20].y]
                            pinky_pip = [Hand_Landmark.landmark[18].x, Hand_Landmark.landmark[18].y]
                            pinky_mcp = [Hand_Landmark.landmark[17].x, Hand_Landmark.landmark[17].y]

                        if gesture_identify(Hand_Landmark.landmark):
                            print('ok')

                        # 手勢開關
                        if not identifier:
                            # 可移動時偵測到OK：關閉移動功能
                            if gesture_identify(Hand_Landmark.landmark):
                                identifier = 1
                                print('pause')
                                time.sleep(1)

                            # 移動&點擊
                            else:
                                # 防靜止晃動的移動量判斷
                                if prev_position is not None:
                                    move_distance = distance_counting(prev_position, (destination_x, destination_y))

                                if move_distance > threshold:
                                    ag.moveTo(destination_x, destination_y)  # 移動鼠標

                                # 紀錄現在位置以供下次迴圈判斷移動量
                                prev_position = (destination_x, destination_y)

                                # 點擊判斷-拇指點擊版本(so far so good)
                                if click_left(thumb_ip, thumb_tip, thumb_mcp, (destination_x, destination_y)):
                                    ag.leftClick(destination_x, destination_y)
                                    time.sleep(0.05)
                                print(count_cos(thumb_ip, thumb_tip, thumb_mcp))

                                if click_right(pinky_pip, pinky_tip, pinky_mcp, (destination_x, destination_y)):
                                    ag.rightClick(destination_x, destination_y)
                                    time.sleep(0.05)

                                # 偵測滾輪的捲動
                                wheel_scroll(mid_pip, mid_tip, mid_mcp)

                        else:
                            # 關閉時偵測到OK：開啟移動功能
                            if gesture_identify(Hand_Landmark.landmark):
                                time.sleep(1)
                                identifier = 0

                        # 將手部識別點繪製於拍攝畫面上
                        for hand_landmarks in results.multi_hand_landmarks:
                            mp_drawing.draw_landmarks(img,
                                                      hand_landmarks,
                                                      mp_hands.HAND_CONNECTIONS,
                                                      mp_drawing_styles.get_default_hand_landmarks_style(),
                                                      mp_drawing_styles.get_default_hand_connections_style())

            cv2.imshow('ROGER BASILISK V4 GAMING MOUSE', img)  # 顯示拍攝畫面
            if cv2.waitKey(5) == ord('q'):
                break
    cam.release()
    cv2.destroyAllWindows


if __name__ == '__main__':
    hand_detect()
```

---

## 待解問題
1. - [ ] 精準點擊
2. - [x] 較好的點擊
3. - [ ] 更好的移動依據(慢來，反正能動就先別改
4. - [x] 移動涵蓋螢幕
5. - [x] 在移動到邊緣時手指仍在鏡頭內
6. - [ ] 鼠標穩定性提升
7. - [ ] 食指操縱點擊
8. - [x] 滾輪功能
9. - [x] 右鍵功能
10. - [x] <font color="5ec653">滑鼠的手勢開關</font>
11. - [x] <font color="fa4cdaw">never gonna give you up. never gonna let you down</font>
12. - [ ] 新環境和角度下的校正功能

## 參考網站

可用參考網站：
1. mediapipe & autogui合用指南 http://www.ctimes.com.tw/DispArt-tw.asp?O=HK75TA82N5CARASTD3
2. 幾乎跟我們要做的東西一樣的中國網站 https://blog.csdn.net/Already8888/article/details/126125467
3. 手部辨識的座標原理 https://blog.csdn.net/weixin_45930948/article/details/115444916
4. 手勢快捷，貌似挺有料 https://medium.com/jimmy-wang/mediapipe-hands-%E6%93%8D%E4%BD%9C%E8%AA%AA%E6%98%8E-%E5%AF%A6%E6%88%B0%E7%AF%87-%E4%B8%8B-e273bda92c48
5. pyautogui 中文教學 https://ithelp.ithome.com.tw/articles/10277668?sc=iThelpR #老哥，這玩意兒可以拿來按popcat :wth
6. mediapipe 手部辨識教學 https://ithelp.ithome.com.tw/m/articles/10299964
7. class教學 https://steam.oxxostudio.tw/category/python/basic/class.html
8. thread教學(平行化執行緒) https://steam.oxxostudio.tw/category/python/library/threading.html
9. hackmd教學 https://hackmd.io/@Kawaii-kanataso/Hackmd_tutorial?fbclid=lwAR3AWX-UWrnqkaBk5PddtFhXl2OFoSkhe0ZuwYfS0Ue00i7lp0gjq_aLglw_aem_th_AZIDMgTMo92k9HBiDPVZOuF1aY8CBgymUL2MelpYhhDFe8T8KwU_1w_abGRwR5T9lwU&mibextid=S66gvF