---
# System prepended metadata

title: Data Cleanup Pipeline

---

# Data Cleanup Pipeline

## Getting Started
**The source audio files for these are [here](https://we.tl/t-PPnTiaRJsD)**

The audio sources and transcriptions are extracted from the following videos:

https://www.youtube.com/watch?v=UOoJe35T2GY
https://www.youtube.com/watch?v=8-BLWt_GdE8
https://www.youtube.com/watch?v=i5Ioa6t47lM
https://www.youtube.com/watch?v=1gqpHZoc3es
https://www.youtube.com/watch?v=5DAwY85_gYQ
https://www.youtube.com/watch?v=fjy5Gk9-ulg
https://www.youtube.com/watch?v=OgiDY_AIFMg
https://www.youtube.com/watch?v=-iPMwOLWoE4
https://www.youtube.com/watch?v=bxBE-DYF-3c
https://www.youtube.com/watch?v=wA3Ta_lPUmM
https://www.youtube.com/watch?v=_H8ru45qmO0
https://www.youtube.com/watch?v=7sJlNObrBRo
https://www.youtube.com/watch?v=oI-l9Ss1aYM

## Audio Pre-Processing
We have a system to generate transcriptions. These transcriptions will need to later be human reviewed.

However, to make sure the transcriptions are accurate and don't include other speakers, we need to pre-process the data.

### Step One - Remove all audio that isn't the speaker
We want to load the full WAV files and delete any noise, leaving the speaker audio in place. The final output of this step should be an audio file of equal length to the input file, but with all of the parts that are not the speaker removed or replaced with silence.

### Step Two - Normalize and equalize all audio tracks together
The speech synthesis works best on audio that is clear and consistent. We want to normalize and equalize the tracks so that they are all the same level and ideally frequency range / tonality. Feel free to add compression, limiting, etc

**If there are any audio files which don't sound like the others or fit the corpus well, feel free to exclude them**

We particularly want to look out for noise, reverb, music, etc as these negatively impact the training

### Step Three - Remove Noise
Audio kind of modern audio noise cleaning tool (Izotope, etc) that can remove external noise is helpful here

### Step Four -- Export as 22khz mono
We will be training on mono sources and want to work with mono exlusively once prepared, since the rest of the process will go faster

### Step Five -- Zip and prepare for transcription
Please send the content back to The Nexus for transcription, along with any notes

# Transcription and Timing

The Nexus will run the transcription from our repo here:
https://github.com/AtlasFoundation/AutomaticTikTalk/tree/main/youtube

If the audio engineer knows python, they can run this themselves. However, we expect that The Nexus will need to run this. Feel free to try it, though!

# Transcription Review

Once transcription is finished, The Nexus will send the metadata.txt and split WAV files containing the transcriptions and timings back to the audio engineer. The audio engineer will need to listen through to all audio and review the transcripts to make sure they are valid.

For words or names, please spell out phonetically as best you can hear if it is not a common word. For example, for the character Kaname Date, we would use 'dah tay' whenever a character said it.

Here is the phonetic annunciation that we use for all words:
http://svn.code.sf.net/p/cmusphinx/code/trunk/cmudict/cmudict-0.7b

All expletives will need to be added if the transcription has removed them. For usage of particularly offensive words, we leave it to the audio engineer's discretion to transcribe or delete all uses from the source audio.

# Final Review

The final dataset that is fed into the training system will be the metadata.csv file and a collection of split audio files, several words each in length.

The audio engineer will need to review the final audio and transcriptions to make sure everything lines up, matches, and sounds relatively close together.