###### tags: `Reading sessions`

[toc]

---

# 2023

<https://arxiv.org/pdf/2210.17546.pdf>

## [Preventing Generation of Verbatim Memorization in Language Models Gives a False Sense of Privacy](https://eprint.iacr.org/2021/934.pdf)
* By Google Research, ETH Zürich, Cornell University

* The paper focues on memorization attacks on Microsoft's Github Copilot (Copilot is a code auto-complete service which is based on an OpenAI model which was trained on GitHub code)
* Copilot uses a filtering mechanism that blocks model outputs from being suggested if they overlap significantly (approximately 150 characters) with a training example.
* The filter can be circumvented by using "style transfer" prompts which cause the model to produce memorized (but not verbatim) output(s).
* "Stlye transfer" prompts in this context can include prompting Copilot with different languges (both natural and programming), and with a different naming convention and comments, etc.
* This paper introduces the term "paraphrased memorization" in addition to the previously considered verbatim definitions. Generally speaking "paraphrased memorization" tries to circumvent verbatim memorization by adding noise to the output i.e. tabs vs spaces, different code comments, different variable names etc.

Take aways:
* Definition of memorization is domain dependent e.g. code vs natural languges.
* Models can leak private code as seen in the Samsung incident <https://www.theregister.com/2023/04/06/samsung_reportedly_leaked_its_own/>
* Memorization can not only lead to privacy but also to copyrigh issues.
* Mechanisms to detect copyright infringement and plagiarism should be considered in order to detect memorization and reduce the output of private and/or copyrighted material.

###### tags: `LLM Memorization`