# PLUG AND PLAY LANGUAGE MODELS: A SIMPLE APPROACH TO CONTROLLED TEXT GENERATION
2023/2/15
###### tags: `Study Group`
**Published as a conference paper at ICLR 2020**
[Reference](https://arxiv.org/pdf/1912.02164.pdf)


## PLUG AND PLAY LANGUAGE MODELS
### 1. LANGUAGE MODELING WITH TRANSFORMERS
- In this paper, we use a transformer to model the distribution of natural language.

### 2. STEERING GENERATION: ASCENDING log $p(a|x)$
- In order to control the output of the language model, at every generation step t,
- we shift the history Ht in the direction of the sum of two gradients:
- one toward higher log-likelihood (LL) of the attribute a under the conditional attribute model *p(a|x)*.
- one toward higher LL of the unmodified language model *p(x)*.
- Combining these factors with a variable multiplier provides us with a controllable “knob” to guide generation in a given direction with a specified strength.


### 3. ENSURING FLUENCY: ASCENDING log p(x)
- Minimize KL divergence between the output distribution of the modified and unmodified language models.
-

### 4. SAMPLING AND RANKING
- In the first step, forward pass, the probability distribution is obtained based on the hidden layer of LM.
- In the second step, backward pass, the hidden layer of LM is updated using the gradient of the attribute model
- In the third step, the new probability distribution is recalculated.
- This process of updating the latents is repeated at each time-step, leading to a gradual transition towards the desired attribute.

[A Multitask, Multilingual, Multimodal Evaluation of ChatGPT
on Reasoning, Hallucination, and Interactivity](https://arxiv.org/pdf/2302.04023v1.pdf)

[Controllable text generation](https://zhuanlan.zhihu.com/p/493321293)
toxicity reducing

- App
- Why
- How