# 電路學期末專題- Low-cost Facial Recognition Doorlock System
## 功能介紹
1. 透過AMB82開發版可連網的特性，可於同區域網內時實查看錄像。
2. 電磁鎖解鎖(開門)，可以透過以下兩種方法。
* 第一種方法為"臉部辨識解鎖"。AMB82本就是Realtek專門設計來跑相關辨識模型的開發版，透過跑臉部辨識模型，將實時影像與資料庫做2D圖像對比，如果比對正確，則LCD會顯示"Welcome Home!"，電磁鎖會開啟3s，並於3s後重新上鎖，如比對錯誤，則不會作動。
* 第二種方法則是"密碼解鎖"。輸入密碼後，會進行密碼比對，若正確會於LCD顯示 "Welcome Home!"，電磁鎖會開啟3s，並於3s後重新上鎖，如比對錯誤則會顯示"Wrong Password!"，並於3s後可重新輸入。
* 電磁鎖開啟時外部綠色LED會亮，關閉時則紅色LED會亮。
* 以上兩種解鎖方法只能擇一進行，不可同時!

## 所需材料
1. Realtek AMB82-mini AI Camera:[link](https://www.icshop.com.tw/product-page.php?29111)


2. Arduino Uno-自備


4. 電磁鎖:[link](https://shopee.tw/%E2%96%BA1528%E2%97%846V-12V%E5%B0%8F%E5%9E%8B%E5%BE%AE%E5%9E%8B%E6%AB%83%E9%96%80%E9%9B%BB%E6%8E%A7%E9%8E%96-%E9%9B%BB%E7%A3%81%E9%8E%96-%E9%9B%BB%E6%8F%92%E9%8E%96-%E9%9B%BB%E5%AD%90%E9%8E%96-i.4877344.2268211356?sp_atk=a4bbb49d-50f4-4e19-b6ee-c2102e5011d2&xptdk=a4bbb49d-50f4-4e19-b6ee-c2102e5011d2)

* 觸發電源大小:6V/0.6A


5. 繼電器:[link](https://shopee.tw/Relay-1%E8%B7%AF-5V-12V-24V-%E7%B9%BC%E9%9B%BB%E5%99%A8%E6%A8%A1%E7%B5%84-%E6%95%B8%E4%BD%8D%E8%A8%8A%E8%99%9F-%E4%B8%80%E4%BD%8D-%E6%90%AD%E8%BC%89%E5%85%89%E8%80%A6-%E9%9B%BB%E6%BA%90%E9%9A%94%E9%9B%A2-%E9%AB%98%E9%9B%BB%E4%BD%8D%E8%A7%B8%E7%99%BC-%E4%BD%8E%E9%9B%BB%E4%BD%8D%E8%A7%B8%E7%99%BC%E5%8F%AF%E8%AA%BF-i.656213378.15057338988?sp_atk=faa60d40-24c8-462b-8467-1e41da22cb2f&xptdk=faa60d40-24c8-462b-8467-1e41da22cb2f)


6. 杜邦線、跳線數條-自備


8. 麵包板-自備


10. 電磁鎖電源供應:[link](https://shopee.tw/%E3%80%90%E7%94%B0%E6%95%85%E9%87%8E%E3%80%91AC-%E6%95%B4%E6%B5%81%E8%AE%8A%E5%A3%93%E5%99%A8%E9%81%A9%E9%85%8D%E5%99%A8-DC-12V-500mA-1A-1.5A-2A-%E9%96%8B%E9%97%9C%E9%9B%BB%E6%BA%90%E4%BE%9B%E6%87%89%E5%99%A8-US-4.0mm-*-i.795509971.19873883251?sp_atk=8eeafa79-884b-41e2-bb80-49c78e874eff&xptdk=8eeafa79-884b-41e2-bb80-49c78e874eff)


9. AMB82 電源供應:行動電源 or 5V 1A 插座

10. DC母頭:[link](https://shopee.tw/%E3%80%8A597%E3%80%8BDC%E6%AF%8D%E9%A0%AD-%E7%9B%A3%E6%8E%A7-%E6%94%9D%E5%BD%B1%E6%A9%9F%E9%9B%BB%E6%BA%90%E6%8E%A5%E9%A0%AD-%E5%85%8D%E7%84%8A%E6%8E%A5-12V%E5%85%AC%E6%8F%92%E9%A0%AD-%E8%9E%BA%E7%B5%B2%E9%9B%BB%E6%BA%90%E6%8F%92%E9%A0%AD-i.105691234.1707075293?sp_atk=5efa8450-7578-425f-97fb-5ce10634ca66&xptdk=5efa8450-7578-425f-97fb-5ce10634ca66)


11. LCD模組with i2c included:[link](https://shopee.tw/LCD1602%E9%A1%AF%E7%A4%BA%E6%A8%A1%E7%B5%84-I2C%E4%BB%8B%E9%9D%A2-16x2%E8%97%8D%E5%BA%95%E7%99%BD%E5%AD%97%E6%B6%B2%E6%99%B6LCD-%E5%B7%B2%E7%84%8A%E5%A5%BDIIC%E8%BD%89%E6%8E%A5%E6%9D%BFPCF8574-%E9%81%A9Arduino-%E6%A8%B9%E8%8E%93-i.4491023.2503817529?sp_atk=090870b5-4a8d-4ce3-955a-1cbbb92cd63f&xptdk=090870b5-4a8d-4ce3-955a-1cbbb92cd63f)


12. 3x3 keypad:[link](https://shopee.tw/%E8%B6%85%E5%A4%A7%E6%8C%89%E9%8D%B5-1*4-3*4-4*4-4*5-%E7%9F%A9%E9%99%A3%E9%8D%B5%E7%9B%A4%E5%A4%96%E6%93%B4%E8%96%84%E8%86%9C%E9%8D%B5%E7%9B%A4%E5%96%AE%E7%89%87%E6%A9%9F%E6%8C%89%E9%8D%B5%E6%A8%A1%E5%A1%8A-i.311618785.22241732309?sp_atk=09e520c3-1aa2-4fe0-8b5d-3b8a13d1ef57&xptdk=09e520c3-1aa2-4fe0-8b5d-3b8a13d1ef57)


13. LED-自備


15. 木板數片

## 電路連接
* 圖片待補
1. Arduino
* LCD
* VCC->Arduino_5V
* GND->Arduino_GND
* SDA->Arduino_A4
* SDL->Arduino_A5
* Relay_Module
* Positive->Arduino_5V
* Negative->Arduino_GND
* S(訊號控制線)->Arduino_13
* COM->電源供應
* NumberPad
* 面相NumberPad，由左至右插在Arduino_5~Arduino_11
2. AMB82
* GND->Arduino_GND
* 8->Arduino_A0
* 3->Led_red
* 4->Led_green
3. 電磁鎖
* 記得先試接，確認作動方向。
* GND與DC母頭連接，正極接在繼電器NO(Normal Open)接口。

## 程式編寫
* 需先設定AMB82環境

<iframe width="560" height="315" src="https://www.youtube.com/embed/PAyg0NEIgYY" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

* 需安裝以下函式庫
* [LiquidCrystal_I2C](https://github.com/fdebrabander/Arduino-LiquidCrystal-I2C-library)
* [Keypad](https://github.com/Chris--A/Keypad)

* 程式碼
* AMB82

    #include "WiFi.h"
    #include "StreamIO.h"
    #include "VideoStream.h"
    #include "RTSP.h"
    #include "NNFaceDetectionRecognition.h"
    #include "VideoStreamOverlay.h"
    #include "AmebaFatFS.h"

    //鏡頭相關腳位定義
    #define CHANNELVID  0 
    #define CHANNELJPEG 1 
    #define CHANNELNN   3 

    //設定臉部辨識解析度
    #define NNWIDTH  576
    #define NNHEIGHT 320

    #define RECTTEXTLAYER0 OSDLAYER0
    #define RECTTEXTLAYER1 OSDLAYER1

    //腳位定義
    #define RED_LED    3
    #define GREEN_LED  4
    #define DOORLOCK  8

    VideoSetting configVID(VIDEO_FHD, 30, VIDEO_H264, 0);
    VideoSetting configJPEG(VIDEO_FHD, CAM_FPS, VIDEO_JPEG, 1);
    VideoSetting configNN(NNWIDTH, NNHEIGHT, 10, VIDEO_RGB, 0);
    NNFaceDetectionRecognition facerecog;
    RTSP rtsp;
    StreamIO videoStreamer(1, 1);
    StreamIO videoStreamerFDFR(1, 1);
    StreamIO videoStreamerRGBFD(1, 1);

    char ssid[] = "HaHaPiYan";  //設定 wifi ssid
    char pass[] = "00000000";   //設定 wifi password
    int status = WL_IDLE_STATUS;

    int resultSize = 0;
    uint32_t img_addr = 0;
    uint32_t img_len = 0;
    String fileName;
    long counter = 0;

    //檔案初始化
    AmebaFatFS fs;

    void setup() {
        pinMode(RED_LED, OUTPUT);
        pinMode(GREEN_LED, OUTPUT);
        pinMode(DOORLOCK, OUTPUT);

        Serial.begin(115200);

        //wifi連線
        while (status != WL_CONNECTED) {
            Serial.print("Attempting to connect to WPA SSID: ");
            Serial.println(ssid);
            status = WiFi.begin(ssid, pass);

            // wait 2 seconds for connection:
            delay(2000);
        }

        //鏡頭初始化
        Camera.configVideoChannel(CHANNELVID, configVID);
        Camera.configVideoChannel(CHANNELJPEG, configJPEG);
        Camera.configVideoChannel(CHANNELNN, configNN);
        Camera.videoInit();

        rtsp.configVideo(configVID);
        rtsp.begin();

        //臉部辨識模型設定
        facerecog.configVideo(configNN);
        facerecog.modelSelect(FACE_RECOGNITION, NA_MODEL, DEFAULT_SCRFD, DEFAULT_MOBILEFACENET);
        facerecog.begin();
        facerecog.setResultCallback(FRPostProcess);

        videoStreamer.registerInput(Camera.getStream(CHANNELVID));
        videoStreamer.registerOutput(rtsp);
        if (videoStreamer.begin() != 0) {
            Serial.println("StreamIO link start failed");
        }

        //開始影片串流
        Camera.channelBegin(CHANNELVID);
        Camera.channelBegin(CHANNELJPEG);

        videoStreamerRGBFD.registerInput(Camera.getStream(CHANNELNN));
        videoStreamerRGBFD.setStackSize();
        videoStreamerRGBFD.setTaskPriority();
        videoStreamerRGBFD.registerOutput(facerecog);
        if (videoStreamerRGBFD.begin() != 0) {
            Serial.println("StreamIO link start failed");
        }

        Camera.channelBegin(CHANNELNN);

        OSD.configVideo(CHANNELVID, configVID);
        OSD.begin();

        digitalWrite(DOORLOCK, LOW);
    }

    void loop() {
        if (Serial.available() > 0) {
          String input = Serial.readString();
          input.trim();

          //基本指令
          if (input.startsWith(String("REG="))) {
            String name = input.substring(4);
            facerecog.registerFace(name);
          } 
          else if (input.startsWith(String("EXIT"))) {
            facerecog.exitRegisterMode();
          } 
          else if (input.startsWith(String("RESET"))) {
            facerecog.resetRegisteredFace();
          } 
          else if (input.startsWith(String("BACKUP"))) {
            facerecog.backupRegisteredFace();
          } 
          else if (input.startsWith(String("RESTORE"))) {
            facerecog.restoreRegisteredFace();
          }
        }

        delay(1000);
        OSD.createBitmap(CHANNELVID,RECTTEXTLAYER0);
        OSD.createBitmap(CHANNELVID,RECTTEXTLAYER1);
        OSD.update(CHANNELVID, RECTTEXTLAYER0);
        OSD.update(CHANNELVID, RECTTEXTLAYER1);
    }

    void FRPostProcess(std::vector<FaceRecognitionResult> results) {
        uint16_t im_h = configVID.height();
        uint16_t im_w = configVID.width();

        printf("Total number of faces detected = %d\r\n", facerecog.getResultCount());

        OSD.createBitmap(CHANNELVID, RECTTEXTLAYER0);
        OSD.createBitmap(CHANNELVID, RECTTEXTLAYER1);
        if (facerecog.getResultCount() > 0) {
            if (1) {
                if (facerecog.getResultCount() > 1) { // Door remain close when more than one face detecteD
                    digitalWrite(RED_LED, HIGH);
                    digitalWrite(GREEN_LED, LOW);
                } else {
                    FaceRecognitionResult singleItem = results[0];
                    if (String(singleItem.name()) == String("unknown")) {  
                        //臉部辨識結果錯誤，紅色警示開啟
                        digitalWrite(RED_LED, HIGH);
                        digitalWrite(GREEN_LED, LOW);
                    } else {
                        //臉部辨識結果正確，綠色通行燈開啟，並進行解鎖訊號傳輸工作
                        digitalWrite(RED_LED, LOW);
                        digitalWrite(GREEN_LED, HIGH);
                        fileName = String(singleItem.name());
                        digitalWrite(DOORLOCK, HIGH);
                        Serial.println("Door Opened!");
                        delay(3000);
                        digitalWrite(DOORLOCK,LOW);
                        Serial.println("Door Locked!");     
                    }
                }
            }
        }

        for (uint32_t i = 0; i < facerecog.getResultCount(); i++) {
            FaceRecognitionResult item = results[i];
            int xmin = (int)(item.xMin() * im_w);
            int xmax = (int)(item.xMax() * im_w);
            int ymin = (int)(item.yMin() * im_h);
            int ymax = (int)(item.yMax() * im_h);

            uint32_t osd_color;
            int osd_layer;
            if (String(item.name()) == String("unknown")) {
                osd_color = OSD_COLOR_RED;
                osd_layer = RECTTEXTLAYER0;
            } else {
                osd_color = OSD_COLOR_GREEN;
                osd_layer = RECTTEXTLAYER1;
            }
            printf("Face %d name %s:\t%d %d %d %d\n\r", i, item.name(), xmin, xmax, ymin, ymax);
            OSD.drawRect(CHANNELVID, xmin, ymin, xmax, ymax, 3, osd_color, osd_layer);

            char text_str[40];
            snprintf(text_str, sizeof(text_str), "Face:%s", item.name());
            OSD.drawText(CHANNELVID, xmin, ymin - OSD.getTextHeight(CHANNELVID), text_str, osd_color, osd_layer);
        }
        OSD.update(CHANNELVID, RECTTEXTLAYER0);
        OSD.update(CHANNELVID, RECTTEXTLAYER1);
    }

* Arduino

    #include "Keypad.h"
    #include <Wire.h>
    #include <LiquidCrystal_I2C.h>

    const byte ROWS = 4; // 列數(橫的)
    const byte COLS = 3; // 行數(直的)

    int i=0, j=0;
    #define LCD_ROWS 2  // LCD顯示器的列數
    #define LCD_COLS 16 // LCD顯示器的行數

    //鍵盤上每一個按鍵的名稱
    char keys[ROWS][COLS] = {
      {'1','2','3'},
      {'4','5','6'},
      {'7','8','9'},
      {'*','0','#'}
    };

    byte rowPins[ROWS] = {11, 10, 9, 8}; //定義列的腳位
    byte colPins[COLS] = {7, 6, 5}; //定義行的腳位

    String passcode = "1234";   // 預設密碼
    String inputCode = "";      // 暫存用戶的按鍵字串
    bool acceptKey = true;      // 是否接受用戶按鍵輸入的boolean value

    int value_A0;

    //初始化鍵盤
    Keypad customKeypad = Keypad( makeKeymap(keys), rowPins, colPins, ROWS, COLS);

    // LCD顯示器
    LiquidCrystal_I2C lcd(0x27, 16, 2);  // 設定模組位址0x27，以及16行, 2列的顯示形式

    void clearRow(byte n) {
      byte last = LCD_COLS - n;
      lcd.setCursor(n, 1); // 移動到第2行，"PIN:"之後

      for (byte i = 0; i < last; i++) {
        lcd.print(" ");
      }
      lcd.setCursor(n, 1);
    }

    //重設LCD顯示文字和輸入狀態。
    void resetLocker() {
      lcd.clear();
      lcd.print("Enter Password");
      lcd.setCursor(0, 1); 
      lcd.print("PIN:");
      lcd.cursor();

      acceptKey = true;
      inputCode = "";
    }

    // 比對用戶輸入的密碼
    void checkPinCode() {
      acceptKey = false;  // 暫時不接受用戶按鍵輸入
      clearRow(0);        // 從第0個字元開始清除LCD畫面
      lcd.noCursor();
      lcd.setCursor(0, 1);
      // 比對密碼
      if (inputCode == passcode) {
        lcd.print("Welcome Home!");
        digitalWrite(13, HIGH);
        delay(3000);
        digitalWrite(13, LOW);
      } 
      else {
        lcd.print("Wrong Password!");
      }
      delay(3000);
      resetLocker();    
    }


    void setup() {
      pinMode(13, OUTPUT);
      pinMode(A0, INPUT);
      digitalWrite(13, LOW);
      Serial.begin(9600);
      lcd.init();       
      lcd.backlight();  
      resetLocker();
    }

    void loop() {
      value_A0=analogRead(A0); //以A0接收amb82開發板的訊號接收
      //Serial.println(value_A0);

      char key = customKeypad.getKey();

      // 數字鍵盤的解鎖
      if (acceptKey && key != NO_KEY) {
        if (key == '*') {   // 清除畫面
          clearRow(4);  // 從第4個字元開始清除
          inputCode = "";
        } else if (key == '#') {  
          checkPinCode(); // 比對輸入密碼
        } else {
          inputCode += key;  // 儲存用戶的按鍵字元
          lcd.print('*');
        }
      }

      // 臉部辨識的解鎖
      if(value_A0>900){
        acceptKey = false;
        lcd.clear();
        lcd.noCursor();
        lcd.print("Welcome Home!");
        digitalWrite(13, HIGH);
        delay(3000);
        digitalWrite(13, LOW);
        resetLocker();
      }
    }

## 遇到的問題及注意事項
1. 一開始我們想使用Esp32來進行臉部辨識，但辨識度、流暢度及穩定度皆不佳。
* 解決方法: 改用AMB82
2. 電腦讀取不到AMB82。
* 解決辦法: 安裝CH340接口的驅動程式([link](https://www.wch.cn/download/CH341SER_EXE.html))
3. Arduino無法讀取到AMB82的訊號。
* 解決辦法: 兩塊開發板間GND要接GND，形成完整迴路。
4. 程式編譯錯誤
* 可能原因
1. 未正確安裝函式庫。
2. 未正確設定開發環境。

## 外觀設計
* 待補

## 成品展示


## 參考資料(References)
* 企劃靈感來源

<iframe width="560" height="315" src="https://www.youtube.com/embed/_VOmfJ4x-Fg" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>

* 元件使用
* KeyPad使用:[link](https://blog.jmaker.com.tw/arduino-keypad-4x4/)
* AMB82使用:[link](https://www.amebaiot.com/zh/amebapro2-amb82-mini-arduino-getting-started/)
* 繼電器接法:[link](https://blog.jmaker.com.tw/arduino-relay/)
* LCD i2c使用:[link](https://crazymaker.com.tw/arduino-lcd-i2c-tutorial/)