# 計算 Discrete Semantic Entropy
> 論文：Detecting hallucinations in large language models using semantic entropy

假設我們詢問 Llama 3.1: 70B 模型 "Who are you?"，並生成以下五個回答：

1. "I'm Llama 3.1: 70B, an AI created by Microsoft. I don't have feelings or personal experiences, but I'm here to assist you with your questions!"
2. "I'm Llama 3.1: 70B, an AI system developed by Microsoft. I'm designed to provide information and help with various tasks."
3. "I am Llama 3.1: 70B, a Microsoft AI model. I don't have emotions or personal experiences but can assist with information."
4. "I’m Llama 3.1: 70B, an AI created by Microsoft to help with information and tasks. I don’t have personal experiences or feelings."
5. "I’m Llama 3.1: 70B, developed by Microsoft. I don't have personal experiences, but I’m here to help you with your inquiries."

## 步驟 1：生成多個答案
- 從模型生成多個對相同問題的回答，這些回答可以通過改變隨機種子或使用不同的采樣策略來獲得。

## 步驟 2：語義聚類（Semantic Clustering）
- 將生成的回答根據語義進行Clustering。假設Clustering結果如下：
- **Cluster A**: Response 1, Response 4, Response 5
- **Cluster B**: Response 2, Response 3

## 步驟 3：計算semantic cluster的概率分佈
- 計算每個semantic cluster的回答比例：
- **Cluster A**: 3/5 = 0.6
- **Cluster B**: 2/5 = 0.4

## 步驟 4：計算 Discrete Semantic Entropy
- 使用 Shannon entropy 公式計算離散語義熵：

- 這個結果 \( H = 0.881 \) 是這組回答的 **discrete semantic entropy**。
- entropy越高，表示模型對此問題的回答在語義上越不確定；
反之，entropy越低，表示模型的回答在語義上較為一致和確定。