generation_config.json
===
###### tags: `LLM / tokenizer`
###### tags: `ML`, `NLP`, `NLU`, `LLM`, `tokenizer`, `Hugging Face`, `AutoTokenizer`, `tokenizer_config.json`, `single_word"`, `lstrip`, `rstrip"`, `normalized`, `special`, `generation_config.json`, `temperature`, `top_k`, `top_p`

<br>

[TOC]

<br>

## 用途
- `generation_config.json` 用於設定文本生成參數，控制模型生成行為及文本品質。
- **備註**：一般 model 常規用法，非 llama3 or vLLM 特有

<br>

## 載入 generation_config.json 的範例

### generation_config.json

{
  "bos_token_id": 0,
  "eos_token_id": [2],
  "do_sample": true,
  "temperature": 0.6,
  "max_length": 4096,
  "top_p": 0.9,
  "transformers_version": "4.40.0.dev0"
}

### demo code

from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

# 加載模型和 tokenizer
tokenizer = AutoTokenizer.from_pretrained("facebook/bart-large-cnn")
model = AutoModelForSeq2SeqLM.from_pretrained("facebook/bart-large-cnn")

# 加載 generation 配置
import json
with open("generation_config.json") as f:
    generation_config = json.load(f)

# 要生成文本的輸入
input_text = "Once upon a time"
inputs = tokenizer(input_text, return_tensors="pt")

# 使用生成配置進行文本生成
output = model.generate(**inputs, **generation_config)
generated_text = tokenizer.decode(output[0], skip_special_tokens=True)

print(generated_text)

- #### 執行結果
- #### `tokenizer.decode(output[0])`
`'</s><s><s>Once upon a time was a time when the world was a very different place. It was also a time of great change, with a lot of very different people in it. The world was changed by the arrival of a group of people known as the "Lost Boys"</s>'`
- #### `tokenizer.decode(output[0], skip_special_tokens=True)`
`'Once upon a time was a time when the world was a very different place. It was also a time of great change, with a lot of very different people in it. The world was changed by the arrival of a group of people known as the "Lost Boys"'`

<br>

## [[huggingface][doc] Text Generation / Generation](https://huggingface.co/docs/transformers/main_classes/text_generation)
- [Parameters / config_file_name](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig.from_pretrained.config_file_name)
- **參數類型**：str or os.PathLike, optional, defaults to "generation_config.json"
- **參數說明**：Name of the generation configuration JSON file to be loaded from pretrained_model_name.
- [Special tokens that can be used at generation time](https://huggingface.co/docs/transformers/main_classes/text_generation#transformers.GenerationConfig.eos_token_id)
- **bos_token_id (int, optional)**
The id of the beginning-of-sequence token.
- **eos_token_id (Union[int, List[int]], optional)**
The id of the end-of-sequence token. Optionally, use a list to set multiple end-of-sequence tokens.

<br>

## [github][huggingface] configuration_utils.py
> [transformers/src/transformers/generation/configuration_utils.py](https://github.com/huggingface/transformers/blob/main/src/transformers/generation/configuration_utils.py)

<br>

## 參考資料
- [基于 transformers 的 generate() 方法实现多样化文本生成：参数含义和算法原理解读](https://blog.csdn.net/muyao987/article/details/125917234)