結合AR與人臉辨識之開源戰鬥力探測器 - David Chang

歡迎來到 https://hackmd.io/c/COSCUP2018 共筆

Image Not Showing Possible Reasons
  • The image file may be corrupted
  • The server hosting the image is unavailable
  • The image path is incorrect
  • The image format is not supported
Learn More →

點擊本頁上方的 開始用 Markdown 一起寫筆記！
手機版請點選上方 按鈕展開議程列表。

請從這裡開始

SourceCode

https://github.com/chechiachang/scouter

投影片

https://github.com/chechiachang/my-speeches/tree/master/fr-ar-open-source-power-detector

Scputer: 3 reasons why

  • Garbage Talks with Linkers
  • Dragon Ball
  • For COSCUP

使用 Unity，放在手機執行

Live Demo

  • 要有人臉+Contribution -> 用github api因為有大家的頭像和其他資料 (Golang)
  • Face Recognition -> Face detection api(Python)
  • API server + database -> Flask + PyMongoDB (Python)
  • Webcam + AR + Face Detection -> Unity + Face Tracker

4 Data Miners (Golang)

  1. User fetcher
  2. User detail fetcher
  3. Avatar downloader
  4. Contribution fetcher

Face detection and face recognition

  1. Detect face from avatars
  2. Detect identities face image
  3. Store identities face image
  4. Detect face from an unknown image
  5. Compare unknown face with matrix, find the distances between all face identities

Github API 限制

  1. API paging limit
    只會回傳前1000筆User資料
  2. API request limit
    Search API 30 query/min
    User API 50k query/hour
  3. Parallel request with Wait Group (Optional)

最簡單的臉部辨識 API (python)

https://github.com/ageitgey/face_recognition
Data pre-processing -> Face Encoding -> Face Recognition

API Server

  1. Consume face image from App
  2. Detect face from image and recognize user by face identity
  3. Get user data from DB and return to App

APP and AR Unity

  • Unity: build app and AR UI
  • OpenCV: image processing library
  • dlib: face recognition tools, models and algorithms

Unity App

  1. Control camera
  2. Detect face on App-side with face tracker
  3. Cut and Send face to API server and get user data back
  4. Display user data to view

Issues

  1. GitHub data source
    Nobody use their won face! 3000human faces/14000 avatars (可愛圖 小動物 嬰兒照等等不能用的資料)
    Github avatar has very low resolution (頭像jpg被壓縮過)

  2. Face recognition API tuning required

  3. I'm a Unity and C# newbie ;)

不是不準 只是正確機率不夠高

Review

用到的技術

  • Golang crawler & html parser
  • Github API
  • Python Flask
  • Face Recogntion API
  • Unity
  • OpenCvForUnity
  • dlib shape predictor
  • C#

#本來會的技術

  • golang

結論

如果你有不會的東西，去報COSCUP，講一個Session就會了

其實台灣的社群對大家很友善，就算Fail了，大家也會很努力協助回饋
選一個有興趣的，成功就賺到，Fail大家笑一笑也很開心

題外話：這個東西超級像大陸的社會信用體系…
不過大陸的是從路上的監視器下手
嘿呀，感覺如果可以拿FB等東西的按贊數之類的也蠻好玩的
劍橋分析事件後，FB Developer現在審核超級嚴苛，我6/1送審聊天機器人到現在還沒下文
居然

tags: COSCUP2018 misc
Select a repo