---
# System prepended metadata

title: MusicLM Study Guide

---

# MusicLM Study Guide

MusicLM ([Paper](https://arxiv.org/pdf/2301.11325.pdf), [Examples](https://google-research.github.io/seanet/musiclm/examples/))

Components:

- SoundStream: E2E neural audio codec
   - [Paper](https://arxiv.org/pdf/2107.03312.pdf)
   - [Talk](https://www.youtube.com/watch?v=V4jj-yhiclk) by first author Neil Zeghidour
   - Residual Vector Quantizer (RVQ)
     - Extension of [VQ-VAE](https://arxiv.org/abs/1711.00937)
     - Inspired by [multistage vector quantization (MSVQ)](https://www.researchgate.net/publication/26501392_Multiple-Description_Multistage_Vector_Quantization)
  - Basis/inspiration for AudioLM
     - Generative audio model via sequence modeling of SoundStream (acoustic) and w2v-BERT (semantic) tokens
     - [Blog](https://ai.googleblog.com/2022/10/audiolm-language-modeling-approach-to.html)
     - [Paper](https://arxiv.org/pdf/2209.03143.pdf)
     - [Examples](https://google-research.github.io/seanet/audiolm/examples/)
- w2v-BERT
  - [Paper](https://arxiv.org/abs/2108.06209)
  - wave2vec 2.0
     - [Paper](https://arxiv.org/pdf/2006.11477.pdf)
     - [Lecture](https://www.youtube.com/watch?v=8Kpowre6yyk) from CU Boulder
  - BERT
     - [Paper](https://arxiv.org/pdf/1810.04805.pdf)
     - [Yannic Kilcher overview](https://www.youtube.com/watch?v=-9evrZnBorM)
- MuLAN: A Joint Embedding of Music Audio and Natural Langauge
   - [Paper](http://www.joonseok.net/papers/mulan.pdf)

