---
# System prepended metadata

title: generation_config.json
tags: [NLU, lstrip, LLM / tokenizer, special, NLP, normalized, LLM, top_k, rstrip", temperature, top_p, tokenizer_config.json, ML, single_word", tokenizer, Hugging Face, generation_config.json, AutoTokenizer]

---

generation_config.json
===
###### tags: `LLM / tokenizer`
###### tags: `ML`, `NLP`, `NLU`, `LLM`, `tokenizer`, `Hugging Face`, `AutoTokenizer`, `tokenizer_config.json`, `single_word"`, `lstrip`, `rstrip"`, `normalized`, `special`, `generation_config.json`, `temperature`, `top_k`, `top_p`

<br>

[TOC]

<br>

## 用途
- `generation_config.json` 用於設定文本生成參數，控制模型生成行為及文本品質。
- **備註**：一般 model 常規用法，非 llama3 or vLLM 特有

<br>

## 載入 generation_config.json 的範例

### generation_config.json
```json=
{
  "bos_token_id": 0,
  "eos_token_id": [2],
  "do_sample": true,
  "temperature": 0.6,
  "max_length": 4096,
  "top_p": 0.9,
  "transformers_version": "4.40.0.dev0"
}
```

### demo code
```python=
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# 加載模型和 tokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn")
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-large-cnn")

# 加載 generation 配置
import json
with open("generation_config.json") as f:
    generation_config = json.load(f)

# 要生成文本的輸入
input_text = "Once upon a time"
inputs = tokenizer(input_text, return_tensors="pt")

# 使用生成配置進行文本生成
output = model.generate(**inputs, **generation_config)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(generated_text)
```
- #### 執行結果
    - #### `tokenizer.decode(output[0])`
        `'</s><s><s>Once upon a time was a time when the world was a very different place. It was also a time of great change, with a lot of very different people in it. The world was changed by the arrival of a group of people known as the "Lost Boys"</s>'`
    - #### `tokenizer.decode(output[0], skip_special_tokens=True)`
        `'Once upon a time was a time when the world was a very different place. It was also a time of great change, with a lot of very different people in it. The world was changed by the arrival of a group of people known as the "Lost Boys"'`
    
<br>

## [[huggingface][doc] Text Generation / Generation](https://huggingface.co/docs/transformers/main_classes/text_generation)
- [Parameters / config_file_name](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig.from_pretrained.config_file_name)
    - **參數類型**：str or os.PathLike, optional, defaults to "generation_config.json"
    - **參數說明**：Name of the generation configuration JSON file to be loaded from pretrained_model_name.
- [Special tokens that can be used at generation time](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig.eos_token_id)
    - **bos_token_id (int, optional)**
        The id of the beginning-of-sequence token.
    - **eos_token_id (Union[int, List[int]], optional)**
        The id of the end-of-sequence token. Optionally, use a list to set multiple end-of-sequence tokens.



<br>

## [github][huggingface] configuration_utils.py
> [transformers/src/transformers/generation/configuration_utils.py](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/configuration_utils.py)

<br>

## 參考資料
- [基于 transformers 的 generate() 方法实现多样化文本生成：参数含义和算法原理解读](https://blog.csdn.net/muyao987/article/details/125917234)

