<h1><center> <img src="https://i.imgur.com/ZUM5sXG.jpg" alt="FuKaKuKKu" title="FuKaKukku" width="1310" height="750" /> <h5>DL4AI FINAL PROJECT PRESENTATION</h5> </center></h1> --- <center> <img src="https://i.imgur.com/WwPgHtG.png" alt="FukkaKukku" width="800" height="450"> </center> <br> <br> - **Project name**: Application of Voice Password and Speaker Verification - **Project goal**: Protect user's information and privacy - **Team member**: . Nguyễn Mạnh Kha 12CTRN . Trần Thiên Phú 12CTRN --- ### Project Description <br> - This AI application is designed to be capable of verifying user's identity, based on the voice characteristics such as tones, features, and at the same time integrating with voice password authentication - **ASR_EncoderDecoder pretrained model** is applied to transcribe speech to text - **ECAPA-TDNN pretrained model** is used extract speaker's embeddings   --- <center> ## TEFPA FRAMEWORK <br> <br> 1. **Task** <br> 1. **Experience** <br> 1. **Function space** <br> 1. **Performance measures** <br> 1. **Algorithm to search/optimize** </center> ---- --- ## TEFPA | | | | --- | --- | |  | <br> **(1) Task:** <br><br> - **Speech-to-text:** With an input of the speaker voice, the model transcribe the audio into text <br> - Use **cosine distance** based on the embedding vectors extracting from audio to recognize the speaker <br><br><br> **(2) Experience**: <br> <br>- **Speech-to-text:** Trained and finetuned on CommonVoice_En <br> - **Speaker verification**: trained on Voxceleb 1+ Voxceleb2 training data. --- ## TEFPA <br> | | | | --------| ----- | |**(3) Function space**: <br>+ 5000-dimensional vector as an output in CNN based ASR_Encoder-Decoder model<br> + 6144x192 sized tensor as an output in ECAPA-TDNN Speaker Recognition model <br><br><br> **(4) Performance measures**: <br> + Word Error Rate (WER): 15.69% <br> + Equal Error Rate (EER): 0.69% <br><br><br>**(5) Algorithm to search and optimize:** <br>+ Connectionist Temporal Classification (CTC) |  --- ## Pretrained model  <img src="https://i.imgur.com/2f51Knt.png" width="610" height="610" /> - [ASR Speech-to-text pretrained model:](https://huggingface.co/speechbrain/asr-wav2vec2-commonvoice-en) A pretrained wav2vec 2.0 model is combined with two DNN layers and finetuned on CommonVoice En. The obtained final acoustic representation is given to the CTC decoder. - [Voice recognition ECAPA-TDNN model:](https://huggingface.co/speechbrain/spkrec-ecapa-voxceleb) Speaker Verification is performed using cosine distance between speaker embeddings. --- # Coding & Demo <br> - Link to [Github_Finalproject_code](https://github.com/Kha1135123/VoiceAuthentication_Finalproject/blob/master/Final_project.py) - Link to [Web_Application](https://share.streamlit.io/kha1135123/voiceauthentication_finalproject/Final_project.py) - Link to [Colab_Notebook](https://colab.research.google.com/drive/1uFdeMSDSDskEbmjGeldM1qVgpD6FsdFd?hl=vi&fbclid=IwAR0Wsj_-tFxCIA2lLQDjz1_5Lls0wGF2AkK7zoBca3ZzuEz7RpEY1M5zpoY#scrollTo=o-iu5aoXCyJd) --- <img src="https://i.imgur.com/Azzskpp.jpg " alt="owari" title="owari" width="900" height="750" > <h4><center> **「おわり」** </h4></center>