# [Week 2] Prompting and Prompt Engineering
[課程目錄](https://areganti.notion.site/Applied-LLMs-Mastery-2024-562ddaa27791463e9a1286199325045c)
[課程連結](https://areganti.notion.site/Week-2-Prompting-and-Prompt-Engineering-57cd1d024ac24819a0c44830a514c41d)
## ETMI5: Explain to Me in 5
:::info
This part of our course content covers the intricacies of prompting and prompt engineering for LLMs. Prompting is the technique of crafting precise instructions to elicit specific responses from LLMs, crucial for their effective use.
:::
:::success
我們課程內容的這一部分涵蓋了對於LLMs提示和提示工程的複雜性。提示是一種制作精確指令以引起LLMs具體響應的技術，這對於有效使用提示至關重要。
:::

:::info
Prompt engineering is an evolving discipline aimed at optimizing these prompts to enhance model performance across various tasks. The importance of prompting lies in its ability to guide LLMs towards producing contextually appropriate and accurate outputs, leveraging their training and understanding of language patterns.
:::
:::success
提示工程是一門不斷發展的學科，旨在最佳化這些提示以增強模型在各種任務中的表現。提示的重要性在於它能夠引導LLMs利用他們的訓練和對語言模式的理解來產生適合上下文的準確輸出。
:::

:::info
The content also touches on the challenges, including potential biases and the risk of hallucination, and the need for techniques to detect and mitigate such issues.
:::
:::success
內容還涉及幾個挑戰，包括潛在的偏見和幻覺風險，以及檢測和緩解此類問題的技術需求。
:::

:::info
Additionally, we briefly go over tools and advanced methods developed for prompt engineering, underscoring the dynamic and collaborative nature of this field in harnessing LLM capabilities.
:::
:::success
此外，我們也簡要地回顧了為提示工程所開發的工具與先進方法，強調了該領域在利用LLMs能力方面的動態和協作性質。
:::
## Introduction
### Prompting
:::info
"**prompting**" refers to the art and science of formulating precise instructions or queries provided to the model to generate desired outputs. It's the input—typically in the form of text—that users present to the language model to elicit specific responses. The effectiveness of a prompt lies in its ability to guide the model's understanding and generate outputs aligned with user expectations.
:::
:::success
"**提示**"是指制定提供給模型以產生所需輸出的精確指令或查詢的藝術和科學。這是使用者向語言模型所呈現的輸入(通常為文字形式)以引發特定的回應。提示的有效性在於它能夠指引模型的理解並生成符合使用者期望的輸出。
:::
### Prompt Engineering
:::info
- Prompt engineering, a rapidly growing field, revolves around refining prompts to unleash the full potential of Language Models in various applications.
- In research, prompt engineering is a powerful tool, enhancing LLMs' performance across tasks like question answering and arithmetic reasoning. Users need to leverage these skills to create effective prompting techniques that seamlessly interact with LLMs and other tools.
- Beyond crafting prompts, prompt engineering is a rich set of skills essential for interacting and developing with LLMs. It's not just about design; it's a crucial skill for understanding and exploiting LLM capabilities, ensuring safety, and introducing novel features like domain knowledge integration.
- This proficiency is vital in aligning AI behavior with human intent. While professional prompt engineers delve into the complexities of AI, the skill isn't exclusive to specialists. Anyone refining prompts for models like ChatGPT is engaging in prompt engineering, making it accessible to users exploring language model potentials.
:::
:::success
- 提示工程，一個快速發展的領域，圍繞著完善提示以釋放語言模型在各種應用中的全部潛力。
- 在研究中，提示工程是一個強大的工具，可以增強LLMs在問答和算術推理等任務中的效能。使用者需要利用這些技能來建立有效的提示工程，與LLMs和其他工具無縫互動。
- 除了制作提示之外，提示工程是一種與LLMs互動和開發必備的豐富技能。這不單純與設計有關；這對於理解與利用LLM的能力、確保安全性以及引入領域知識整合等新穎功能來說是至關重要要技能。
- 這種熟練程度對於讓人工智慧的行為與人類意圖一致來說是重要的。雖然專業的提示工程師深入研究人工智慧的複雜性，但這項技能並非專家獨有的。任何為ChatGPT等模型優化提示的人都在從事提示工程，這讓探索語言模型潛力的使用者都可以使用它。
:::

Image Source: https://zapier.com/blog/prompt-engineering/

## Why Prompting?
:::info
Large language models are trained through a process called unsupervised learning on vast amounts of diverse text data. During training, the model learns to predict the next word in a sentence based on the context provided by the preceding words. This process allows the model to capture grammar, facts, reasoning abilities, and even some aspects of common sense.
:::
:::success
大型語言模型是透過一種稱為無監督學習的過程在大量不同的文本資料上進行訓練的。在訓練過程中，模型學習根據前面單字所提供的上下文來預測句子中的下一個單字。這個過程允許模型捕捉文法、事實、推理能力，甚至是部份常識。
:::

:::info
Prompting is a crucial aspect of using these models effectively. Here's why prompting LLMs the right way is essential:

1. **Contextual Understanding:** LLMs are trained to understand context and generate responses based on the patterns learned from diverse text data. When you provide a prompt, it's crucial to structure it in a way that aligns with the context the model is familiar with. This helps the model make relevant associations and produce coherent responses.
2. **Training Data Patterns:** During training, the model learns from a wide range of text, capturing the linguistic nuances and patterns present in the data. Effective prompts leverage this training by incorporating similar language and structures that the model has encountered in its training data. This enables the model to generate responses that are consistent with its learned patterns.
3. **Transfer Learning:** LLMs utilize transfer learning. The knowledge gained during training on diverse datasets is transferred to the task at hand when prompted. A well-crafted prompt acts as a bridge, connecting the general knowledge acquired during training to the specific information or action desired by the user.
4. **Contextual Prompts for Contextual Responses:** By using prompts that resemble the language and context the model was trained on, users tap into the model's ability to understand and generate content within similar contexts. This leads to more accurate and contextually appropriate responses.
5. **Mitigating Bias:** The model may inherit biases present in its training data. Thoughtful prompts can help mitigate bias by providing additional context or framing questions in a way that encourages unbiased responses. This is crucial for aligning model outputs with ethical standards.
:::
:::success
提示是有效使用這些模型的一個重要觀點。下面說明為什麼以正確的方式提示LLMs非常重要：

1. **上下文理解：** LLMs的訓練是基於從大量文本資料中所學習到的模式來理解上下文並生成響應。當你提供提示時，用與模型所熟悉的上下文一致的方式來建立提示是很重要的。這有助於模型建立相關關聯並產生連貫的響應。
2. **訓練資料模式：** 在訓練過程中，模型從各種文本中學習，捕捉資料中存在的語言細微差別和模式。有效的提示透過合併模型在訓練資料中遇到的類似的語言與結構來利用這種訓練。這使得模型能夠生成與所學習到的模式一致性的響應。
3. **遷移學習：** LLMs利用遷移學習。在不同資料集的訓練過程中所獲得的知識會在提示時遷移到手頭上任務。一個精心設計的提示是可作為橋樑的，連結著訓練期間獲得的通用知識與使用者所需的特定信息或操作。
4. **情境回應的上下文提示：** 透過使用類似於模型訓練所使用的語言和上下文的提示，使用者可以掌握到模型在相似的上下文中理解和生成內容的能力。這會帶來更準確、更適合上下文的響應。
5. **減輕偏差：** 模型也許會繼承其訓練資料中所存在的偏差。考慮周全的提示有助於減少偏差，可以透過提供額外的上下文或鼓勵無偏差響應的方式提出問題。這對於讓模型的輸出合乎道德標準是很重要的。
:::

:::info
To summarize, the training of LLMs involves learning from massive datasets, and prompting is the means by which users guide these models to produce useful, relevant, and policy-compliant responses. It's a collaborative process where users and models work together to achieve the desired outcome. There’s also a growing field called adversarial prompting which involves intentionally crafting prompts to exploit weaknesses or biases in a language model, with the goal of generating responses that may be misleading, inappropriate, or showcase the model's limitations. Safeguarding models from providing harmful responses is a challenge that needs to be solved and is an active research area.
:::
:::success
總的來說，LLMs的訓練涉及從海量資料集中學習，而提示就是讓使用者指導這些模型產生有用、相關且符合政策的響應的手段。這是一個協作過程，使用者與模型共同努力實現所期望的結果。還有一個發展中的領域，稱為對抗性提示(adversarial prompting)，這涉及故意制作提示以利用語言模型中的弱點或偏見，其目標是生成可能具有誤導性、不恰當或展示模型侷限性的響應。保護模型免於提供有害的響應是一個的極需解決的挑戰，也是一個當前的研究領域。
:::
## Prompting Basics
:::info
The basic principles of prompting involve the inclusion of specific elements tailored to the task at hand. These elements include:

1. **Instruction:** Clearly specify the task or action you want the model to perform. This sets the context for the model's response and guides its behavior.
2. **Context:** Provide external information or additional context that helps the model better understand the task and generate more accurate responses. Context can be crucial in steering the model towards the desired outcome.
3. **Input Data:** Include the input or question for which you seek a response. This is the information on which you want the model to act or provide insights.
4. **Output Indicator:** Define the type or format of the desired output. This guides the model in presenting the information in a way that aligns with your expectations.
:::
:::success
提示的基本原則涉及包含當前任務量身定制的特定元素。這些元素包括：

1. **說明：** 明確指定你希望模型執行的任務或動作。這為模型的響應設定了上下文並指導其行為。
2. **上下文：** 提供外部信息或額外的上下文，這可以幫助模型更好地理解任務並生成更準確的響應。上下文對於操控模型往預期的結果是很重要的。
3. **輸入資料：** 包括你所要尋求的答案的輸入或問題。這是你希望模型採取行動或提供見解的信息。
4. **輸出指示器：** 定義所期望的輸出類型或格式。這會指導模型以符合你所期望的方式來呈現信息。
:::

:::info
Here's an example prompt for a text classification task:
Prompt:

Classify the text into neutral, negative, or positive
Text: I think the food was okay.
Sentiment:

:::
:::success
下面給出文本分類任務的提示範本：
Prompt：

Classify the text into neutral, negative, or positive
Text: I think the food was okay.
Sentiment:

:::

:::info
In this example:

- **Instruction:** "Classify the text into neutral, negative, or positive."
- **Input Data:** "I think the food was okay."
- **Output Indicator:** "Sentiment."
:::
:::success
在這個範例中：
- **說明：** "Classify the text into neutral, negative, or positive."
- **輸入數據：** "I think the food was okay."
- **輸出指標：** "Sentiment."
:::

:::info
Note that this example doesn't explicitly use context, but context can also be incorporated into the prompt to provide additional information that aids the model in understanding the task better.
:::
:::success
注意到，這個範例並沒有明確地使用上下文，不過，上下文也可以以合併到提示中的方式來提供額外的信息，幫助模型更好地理解任務。
:::

:::info
It's important to highlight that not all four elements are always necessary for a prompt, and the format can vary based on the specific task. The key is to structure prompts in a way that effectively communicates the user's intent and guides the model to produce relevant and accurate responses.
:::
:::success
這邊要強調是，不過每個提示都一定要把這四個元素寫好寫滿，而且格式的部份也可能會根據具體任務而有所不同。關鍵是以有效地傳達使用者的意圖並且指導模型產生相關且準確的響應的方式來建立提示。
:::

:::info
OpenAI has recently provided guidelines on best practices for prompt engineering using the OpenAI API. For a detailed understanding, you can explore the guidelines [here](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api), the below points gives a brief summary:

1. **Use the Latest Model:** For optimal results, it is recommended to use the latest and most capable models.
2. **Structure Instructions:** Place instructions at the beginning of the prompt and use ### or """ to separate the instruction and context for clarity and effectiveness.
3. **Be Specific and Descriptive:** Clearly articulate the desired context, outcome, length, format, style, etc., in a specific and detailed manner.
4. **Specify Output Format with Examples:** Clearly express the desired output format through examples, making it easier for the model to understand and respond accurately.
5. **Use Zero-shot, Few-shot, and Fine-tune Approach:** Begin with a zero-shot approach, followed by a few-shot approach (providing examples). If neither works, consider fine-tuning the model.
6. **Avoid Fluffy Descriptions:** Reduce vague and imprecise descriptions. Instead, use clear instructions and avoid unnecessary verbosity.
7. **Provide Positive Guidance:** Instead of stating what not to do, clearly state what actions should be taken in a given situation, offering positive guidance.
8. **Code Generation Specific - Use "Leading Words":** When generating code, utilize "leading words" to guide the model toward a specific pattern or language, improving the accuracy of code generation.
:::
:::success
OpenAI最近提供了有關使用OpenAI API進行提示工程的最佳實踐指南。更多細節，你可以探索[此處](https://help.openai.com/en/articles/6654000-best-practices-for-prompt-engineering-with-openai-api)的指南，下面幾點給一個簡短的總結：

1. **使用最新模型：** 為了得到最佳結果，建議使用最新最強的模型。
2. **結構說明：** 將說明放在提示的開頭，並使用###或"""分隔說明和上下文，以確認清晰度和有效性。
3. **具體且具描述性：** 以具體且詳細的方式清楚地闡述所期望的上下文、結果、長度、格式、風格等。
4. **透過範例指定輸出格式：** 透過範例清楚地表達所期望的輸出格式，使模型更容易理解並準確回應。
5. **使用Zero-shot、Few-shot和Fine-tune的方法：** 從Zero-shot方法開始，然後是Few-shot方法(提供範例)。如果兩種方法都不起作用，那就只好考慮微調模型。
6. **避免模糊的描述：** 減少含糊和不精確的描述。相反，請使用清晰的說明並避免不必要的冗長。
7. **提供正向指導：** 不要跟模型說不要做什麼，而是明確地說明在給定的情況下應採取哪些動作，提供正向指導。
8. **程式碼產生特定 - 使用"引導詞"：** 在生成程式碼的時候，利用"引導詞"來引導模型走向特定模式或語言，提高程式碼生成的準確性。
:::

:::info
💡It’s also important to note that crafting effective prompts is an iterative process, and you may need to experiment to find the most suitable approach for your specific use case. Prompt patterns may be specific to models and how they were trained (architecture, datasets used etc.)
:::
:::success
💡同樣重要的是要注意，制作有效的提示是一個迭代過程，你可能需要做點實驗才能找到最適合你的特定情況的方法。提示模式可能特定於模型及其訓練方式(架構、使用的資料集等)，也就是會根據模型以及訓練方式而有所不同。
:::

:::info
Explore these examples of prompts to gain a better understanding of how to craft effective prompts in different use-cases.
:::
:::success
探索這些提示範例，以便更好地瞭解如何在不同的使用情況下制作有效的提示。
:::

## Advanced Prompting Techniques
:::info
Prompting techniques constitute a rapidly evolving area of research, with researchers continually exploring novel methods to effectively prompt models for optimal performance. The simplest forms of prompting include zero-shot, where only instructions are provided, and few-shot, where examples are given, and the language model (LLM) is tasked with replication. More intricate techniques are elucidated in various research papers. While the provided list is not exhaustive, existing prompting methods can be tentatively classified into high-level categories. It's crucial to note that these classes are derived from current techniques and are not exhaustive or definitive; they are subject to evolution and modification, reflecting the dynamic nature of advancements in this field. It's important to highlight that numerous methods may fall into one or more of these classes, exhibiting overlapping characteristics to get the benefits offered by multiple categories.
:::
:::success
提示技術構成了一個快速發展的研究領域，研究人員不眠不休地探索新的方法來有效地提示模型以獲得最佳性能。最簡單的提示形式包括Zero-shot(僅提供說明)和Few-shot(給出範例)，而語言模型(LLM)的任務就是複製。更複雜的技術在各種研究論文中有詳細說明。儘管我所提供的清單並不是全部，不過，現有的提示方法可以暫時被分類為高級類別。值得注意的是，這些類別是源於目前的技術，並非詳盡的或確定性的；它們會不斷演變和修改，反映了這個領域進步的動態性質。需要強調的是，許多方法可能屬於這些類別中的一個或多個，表現出重疊的特徵以獲得多個類別所提供的好處。
:::

### A. Step**-by-Step Modular Decomposition**
:::info
These methods involve breaking down complex problems into smaller, manageable steps, facilitating a structured approach to problem-solving. These methods guide the LLM through a sequence of intermediate steps, allowing it to focus on solving one step at a time rather than tackling the entire problem in a single step. This approach enhances the reasoning abilities of LLMs and is particularly useful for tasks requiring multi-step thinking.
:::
:::success
這些方法涉及將複雜的問題分解為較小的、可管理的步驟，促使成為一種解決問題的結構化方法。。這些方法引導LLMs完成一系列的中間步驟，使其能夠專注於一次解決一個步驟，而不是一步解決整個問題。這種方法增強了LLMs的推理能力，對於需要多步驟思考的任務特別有用。
:::

:::info
Examples of methods falling under this category include:

1. **Chain-of-Thought (CoT) Prompting:**

Chain-of-Thought (CoT) Prompting is a technique to enhance complex reasoning capabilities through intermediate reasoning steps. This method involves providing a sequence of reasoning steps that guide a large language model (LLM) through a problem, allowing it to focus on solving one step at a time.
:::
:::success
屬於這種類別的方法的範例包括：

1. **Chain-of-Thought (CoT) Prompting：**

思維鏈(CoT)提示是一種透過中間推理步驟來增強複雜推理能力的技術。這種方法涉及提供一系列的推理步驟來指導大型語言模型(LLM)解決問題，使其能夠一次專注於解決一個步驟。
:::
:::warning
Chain-of-Thought：關聯思考、思維鏈都有人使用
:::

:::info
In the provided example below, the prompt involves evaluating whether the sum of odd numbers in a given group is an even number. The LLM is guided to reason through each example step by step, providing intermediate reasoning before arriving at the final answer. The output shows that the model successfully solves the problem by considering the odd numbers and their sums.
:::
:::success
在下面提供的範例中，提示涉及評估在給定的群組中的奇數總和是否為偶數。LLMs一步一步的引導每個範例進行推理，在得出最終答案之前提供中間的推理過程。輸出表明了該模型透過考慮奇數及其總和成功地解決了這個問題。
:::
Image Source: Wei et al. (2022)

:::info
1a. **Zero-shot/Few-Shot CoT Prompting:**

Zero-shot involves adding the prompt "Let's think step by step" to the original question to guide the LLM through a systematic reasoning process. Few-shot prompting provides the model with a few examples of similar problems to enhance reasoning abilities. These CoT methods prompt significantly improves the model's performance by explicitly instructing it to think through the problem step by step. In contrast, without the special prompt, the model fails to provide the correct answer.
:::
:::success
1a. **Zero-shot/Few-Shot CoT Prompting：**

Zero-shot涉及在原始問題中添加"讓我們一步一步思考"的提示，以指導LLMs進行系統的推理過程。 Few-shot提示為模型提供了一些類似問題的範例，以增強推理能力。這些CoT方法透過明確地指示模型逐步思考問題的方式，顯著地提高了模型的效能。相反的，在沒有特殊提示情況下，模型無法提供正確的答案。
:::
Image Source: Kojima et al. (2022)

:::info
1b. **Automatic Chain-of-Thought (Auto-CoT):**

Automatic Chain-of-Thought (Auto-CoT) was designed to automate the generation of reasoning chains for demonstrations. Instead of manually crafting examples, Auto-CoT leverages LLMs with a "Let's think step by step" prompt to automatically generate reasoning chains one by one.
:::
:::success
1b. **Automatic Chain-of-Thought (Auto-CoT)：**

自動化思維鏈(Auto-CoT)設計的目的是自動生成用於演示的推理鏈。 Auto-CoT不是手動制作範例，而是利用LLMs的"讓我們一步一步思考"提示來自動生成推理鏈。
:::
Image Source: Zhang et al. (2022)

:::info
The Auto-CoT process involves two main stages:

1. **Question Clustering:** Partition questions into clusters based on similarity.
2. **Demonstration Sampling:** Select a representative question from each cluster and generate its reasoning chain using Zero-Shot-CoT with simple heuristics.
:::
:::success
Auto-CoT的流程涉及兩個主要階段：

1. **Question Clustering：** 根據相似性將問題分割為叢集(clusters)。
2. **Demonstration Sampling：** 從每個叢集中選擇一個代表性問題，然後使用Zero-Shot-CoT搭配簡單的啟發式方法產生其推理鏈。
:::

:::info
The goal is to eliminate manual efforts in creating diverse and effective examples. Auto-CoT ensures diversity in demonstrations, and the heuristic-based approach encourages the model to generate simple yet accurate reasoning chains.
:::
:::success
目標是消除手動建立多樣且有效範例的工作。Auto-CoT確保展示的多樣性，基於啟發式的方法鼓勵模型生成簡單而準確的推理鏈。
:::

:::info
Overall, these CoT prompting techniques showcase the effectiveness of guiding LLMs through step-by-step reasoning for improved problem-solving and demonstration generation.
:::
:::success
總的來說，這些CoT提示技術展示了透過指導LLMs逐步推理來改進問題解決和演示生成的有效性。
:::

:::info
2. **Tree-of-Thoughts (ToT) Prompting**

Tree-of-Thoughts (ToT) Prompting is a technique that extends the Chain-of-Thought approach. It allows language models to explore coherent units of text ("thoughts") as intermediate steps towards problem-solving. ToT enables models to make deliberate decisions, consider multiple reasoning paths, and self-evaluate choices. It introduces a structured framework where models can look ahead or backtrack as needed during the reasoning process. ToT Prompting provides a more structured and dynamic approach to reasoning, allowing language models to navigate complex problems with greater flexibility and strategic decision-making. It is particularly beneficial for tasks that require comprehensive and adaptive reasoning capabilities.
:::
:::success
2. **Tree-of-Thoughts (ToT) Prompting**

思維樹(ToT)提示是一種擴展思維鏈方法的技術。它允許語言模型探索連貫的文本單元("思維")作為解決問題的中間步驟。 ToT讓模型能夠做出深思熟慮的決策、考慮多種推理路徑並自我評估選擇。它引入了一個結構化框架，其模型可以在推理過程中根據需求向前或回溯。ToT Prompting提供了一種更結構化與動態的推理方法，使語言模型能夠以更大的靈活性與策略決策來克服複雜的問題。它對於需要全面性與自適應推理能力的任務特別有好處。
:::
:::warning
navigate：查詢[報章英文成語教室](https://www.facebook.com/gainwind/posts/%E4%B8%80%E5%80%8B%E4%B8%8D%E5%AE%B9%E6%98%93%E7%BF%BB%E8%AD%AF%E7%9A%84%E6%96%B0%E8%81%9E%E8%8B%B1%E6%96%87%E7%94%A8%E8%AA%9E-navigate-vt%E8%88%8A%E8%AA%9E%E6%84%8F1-%E9%A7%95%E9%A7%9B%E8%88%B9%E8%88%B6%E9%A3%9B%E6%A9%9F%E7%AD%89%E5%B0%8E%E8%88%AA2-%E4%BA%BA%E8%88%B9%E7%AD%89%E8%88%AA%E8%A1%8C%E6%96%BC3-%E7%BE%8E%E5%9C%8B%E7%A9%A9%E5%AE%9A%E5%9C%B0%E7%AD%86%E7%9B%B4%E8%B5%B0%E4%BD%BF%E9%80%9A%E9%81%8E%E8%AD%B0%E6%A1%88%E7%AD%89%E6%96%B0%E8%AA%9E%E6%84%8F1-%E6%A9%AB%E6%B8%A1-now-a-ne/1019842624816337/?locale=zh_TW)
:::

:::info
**Key Characteristics:**

- **Coherent Units ("Thoughts"):** ToT prompts LLMs to consider coherent units of text as intermediate reasoning steps.
- **Deliberate Decision-Making:** Enables models to make decisions intentionally and evaluate different reasoning paths.
- **Backtracking and Looking Ahead:** Allows models to backtrack or look ahead during the reasoning process, providing flexibility in problem-solving.
:::
:::success
**關鍵特徵：**

- **Coherent Units ("Thoughts")：** ToT prompts LLMs將文本的coherent units視為中間推理步驟。
- **Deliberate Decision-Making：** 讓模型能夠有意義地做出決策並評估不同的推理路徑。
- **Backtracking and Looking Ahead：** 讓模型在推理過程中回溯或向前看，為解決問題提供靈活性。
:::
:::warning
Coherent Units：相關、一致、耦合、連貫單位？
:::
Image Source: Yao et el. (2023)
:::info
3. **Graph of Thought Prompting**

This work arises from the fact that human thought processes often follow non-linear patterns, deviating from simple sequential chains. In response, the authors propose Graph-of-Thought (GoT) reasoning, a novel approach that models thoughts not just as chains but as graphs, capturing the intricacies of non-sequential thinking.
:::
:::success
3. **Graph of Thought Prompting**

這項研究源於這樣一個事實，也就是人類的思維過程通常遵循非線性模式，偏離簡單的順序鏈。作為回應，作者提出了思維圖(GoT)推理，這是一種新穎的方法，模型的思維不再是鏈，而是圖，以此捕捉非順序思維的複雜性。
:::

:::info
This extension introduces a paradigm shift in representing thought units. Nodes in the graph symbolize these thought units, and edges depict connections, presenting a more realistic portrayal of the complexities inherent in human cognition. Unlike traditional trees, GoT employs Directed Acyclic Graphs (DAGs), allowing the modeling of paths that fork and converge. This divergence provides GoT with a significant advantage over conventional linear approaches.
:::
:::success
這榑擴展在表示思維單元中引入[典範位移](https://terms.naer.edu.tw/detail/8650e52cb71f6d40e32b9ded1c29744a/)。圖(graph)中的節點象徵著這些思維單元，邊緣則是描繪著連結，給出一個關於人類認知固有的複雜性更真實的描寫。與傳統的樹(trees)不同，GoT採用有向非循環圖(DAG)，允許對分叉和匯聚的路徑進行建模。這種差異給了GoT不同於傳統線性方法的顯著優勢。
:::

:::info
The GoT reasoning model operates in a two-stage framework. Initially, it generates rationales, and subsequently, it produces the final answer. To facilitate this, the model leverages a Graph-of-Thoughts encoder for representation learning. The integration of GoT representations with the original input occurs through a gated fusion mechanism, enabling the model to combine both linear and non-linear aspects of thought processes.
:::
:::success
GoT推理模型在兩階段框架中運作。最一開始，它產生[基本原理](https://terms.naer.edu.tw/detail/74f3ea14a82814f61f150fd087270d08/)，隨後，它產生最終答案。為了促成這一點，該模型利用Graph-of-Thoughts編碼器來做表示學習。 GoT的表示透過gated fusion mechanism(門控融合機制？)與原始輸入進行整合，讓模型能夠思考過程中結合線性和非線性。
:::
Image Source: Yao et el. (2023)
### B. Comprehensive **Reasoning and Verification**

:::info
Comprehensive Reasoning and Verification methods in prompting entail a more sophisticated approach where reasoning is not just confined to providing a final answer but involves generating detailed intermediate steps. The distinctive aspect of these techniques is the integration of a self-verification mechanism within the framework. As the LLM generates intermediate answers or reasoning traces, it autonomously verifies their consistency and correctness. If the internal verification yields a false result, the model iteratively refines its responses, ensuring that the generated reasoning aligns with the expected logical coherence. These checks contributes to a more robust and reliable reasoning process, allowing the model to adapt and refine its outputs based on internal validation
:::
:::success
在提示中的綜合推理和驗證方法需要更複雜的方法，其中推理不僅限於提供最終答案，還涉及生成詳細的中間步驟。這些技術的獨特之處在於框架內整合了自我驗證的機制。當LLMs生成中間答案或推理軌跡時，它會自動驗證它們的一致性和正確性。如果內部驗證產生一個錯誤的結果，模型就會迭代地完善其響應，確保生成的推理符合預期的邏輯一致性。這些檢核有助於實現更穩健和可靠的推理過程，使模型能夠根據內部驗證調整並和完善其輸出。
:::

:::info
1. **Automatic Prompt Engineer**

Automatic Prompt Engineer (APE) is a technique that treats instructions as programmable elements and seeks to optimize them by conducting a search across a pool of instruction candidates proposed by an LLM. Drawing inspiration from classical program synthesis and human prompt engineering, APE employs a scoring function to evaluate the effectiveness of candidate instructions. The selected instruction, determined by the highest score, is then utilized as the prompt for the LLM. This automated approach aims to enhance the efficiency of prompt generation, aligning with classical program synthesis principles and leveraging the knowledge embedded in large language models to improve overall performance in producing desired outputs.
:::
:::success
1. **Automatic Prompt Engineer**

Automatic Prompt Engineer(APE)是一種將指令視為可程式元素的技術，並透過在LLMs提出的候選指令池中進行搜尋來尋求指令的最佳化。APE從經典程式合成和人類提示工程中提取靈感，採用評分函數(scoring function)來評估候選指令的有效性。然後，根據最高分所確定的指令將被用作LLMs的提示。這種自動化方法主要在於增加提示生成的效率，與經典程式合成原理保持一致，並利用大型語言模型中嵌入的知識來提高產生所需輸出的整體效能。
:::
Image Source: Zhou et al., (2022)
:::info
2. **Chain of Verification (CoVe)**

The Chain-of-Verification (CoVe) method addresses the challenge of hallucination in large language models by introducing a systematic verification process. It begins with the model drafting an initial response to a user query, potentially containing inaccuracies. CoVe then plans and poses independent verification questions, aiming to fact-check the initial response without bias. The model answers these questions, and based on the verification outcomes, generates a final response, incorporating corrections and improvements identified through the verification process. CoVe ensures unbiased verification, leading to enhanced factual accuracy in the final response, and contributes to improved overall model performance by mitigating the generation of inaccurate information.
:::
:::success
2. **驗證鏈 (CoVe)**

驗證鏈(Cove)方法透過引入系統驗證過程來解決大型語言模型中的幻覺挑戰。首先，模型會起草一個給使用者查詢的初始回應，這可能會包含不準確的地方。然後，Cove會規劃並提出獨立的驗證問題，主要是不帶偏見地對最初的回應做事實核查。該模型回答這些問題，並根據驗證結果生成最終的響應，然後合併通過驗證過程所確定的更正和改進。Cove確保了無偏的驗證，從而增強最終響應的事實準確性，然後再透過減少不準確信息的生成來提高整體模型的效能。
:::
Image Source: Dhuliawala et al.2023

:::info
3. **Self Consistency**

Self Consistency represents a refinement in prompt engineering, specifically targeting the limitations of naive greedy decoding in chain-of-thought prompting. The core concept involves sampling multiple diverse reasoning paths using few-shot CoT and leveraging the generated responses to identify the most consistent answer. This method aims to enhance the performance of CoT prompting, particularly in tasks that demand arithmetic and commonsense reasoning. By introducing diversity in reasoning paths and prioritizing consistency, Self Consistency contributes to more robust and accurate language model responses within the CoT framework.
:::
:::success
3. **自我一致性**

自我一致性代表了提示工程的一種改進，特別是針對思維鏈提示中naive greedy decoding的侷限性。其核心概念涉及使用few-shot CoT對多個不同的推理路徑進行採樣，並利用生成的響應來識別最一致的答案。此方法旨在增強oT prompting的效能，特別是在那種需要四則運算和常識推理的任務中。透過在推理路徑中引入多樣性與優先考慮一致性，自我一致性有助於在CoT框架內實現更穩健、更準確的語言模型響應。
:::
Image Source: Wang et al. (2022)
:::info
4. **ReACT**

The ReAct framework combines reasoning and action in LLMs to enhance their capabilities in dynamic tasks. The framework involves generating both verbal reasoning traces and task-specific actions in an interleaved manner. ReAct aims to address the limitations of models, like chain-of-thought , that lack access to the external world and can encounter issues such as fact hallucination and error propagation. Inspired by the synergy between "acting" and "reasoning" in human learning and decision-making, ReAct prompts LLMs to create, maintain, and adjust plans for acting dynamically. The model can interact with external environments, such as knowledge bases, to retrieve additional information, leading to more reliable and factual responses.
:::
:::success
4. **ReACT**

ReAct框架結合LLMs中的推理與動作以增強他們在動態任務中的能力。這個框架涉及以交錯的方式生成語言推理軌跡和特定任務的動作。ReAct主要為了解決模型的侷限性，例如思維鏈模型，這些模型缺乏與外部世界的聯繫，並且可能遇到事實幻覺和錯誤傳播等問題。受到人類學習和決策中「行動」和「推理」之間的協同合作的啟發，ReAct促使LLMs動態地建立、維護和調整行動計劃。這模型可以與外部環境(像是知識庫)交互，檢索額外的信息，從而產生更可靠和更真實的響應。
:::
Image Source: Yao et al., 2022
:::info
**How ReAct Works:**

1. **Dynamic Reasoning and Acting:** ReAct generates both verbal reasoning traces and actions, allowing for dynamic reasoning in response to complex tasks.
2. **Interaction with External Environments:** The action step enables interaction with external sources, like search engines or knowledge bases, to gather information and refine reasoning.
3. **Improved Task Performance:** The framework's integration of reasoning and action contributes to outperforming state-of-the-art baselines on language and decision-making tasks.
4. **Enhanced Human Interpretability:** ReAct leads to improved human interpretability and trustworthiness of LLMs, making their responses more understandable and reliable.
:::
:::success
**ReAct 的工作原理：**

1. **動態推理和行動：** ReAct生成語言推理軌跡和動作，允許動態推理以響應複雜的任務。
2. **與外部環境交互：** action step可以與外部來源(像是搜尋引擎或知識庫)交互，以收集信息並完善推理。
3. **提高任務效能：** 此框架整合了推理和動作，這有助於在語言和決策任務方面超越最先進的基線。
4. **增強人類可解釋性：** ReAct提高了LLMs的人類可解釋性和可信度，使他們的響應更容易理解和可靠。
:::
### C. Usage of External Tools/Knowledge or Aggregation
:::info
This category of prompting methods encompasses techniques that leverage external sources, tools, or aggregated information to enhance the performance of LLMs. These methods recognize the importance of accessing external knowledge or tools for more informed and contextually rich responses. Aggregation techniques involve harnessing the power of multiple responses to enhance the robustness. This approach recognizes that diverse perspectives and reasoning paths can contribute to more reliable and comprehensive answers. Here's an overview:
:::
:::success
這類提示方法包括利用外部資源、工具或匯總信息來提高LLMs效能的技術。這些方法認識到獲取外部知識或工具以獲得更見多識廣和上下文豐富的響應的重要性。聚合技術涉及利用多個響應的能力來增強穩健性。這種方法認識到不同的觀點和推理路徑可以有助於獲得更可靠和更全面性的答案。概述如下：
:::

:::info
1. **Active Prompting (Aggregation)**

Active Prompting was designed to enhance the adaptability LLMs to various tasks by dynamically selecting task-specific example prompts. Chain-of-Thought methods typically rely on a fixed set of human-annotated exemplars, which may not always be the most effective for diverse tasks. Here's how Active Prompting addresses this challenge:
1. **Dynamic Querying:**
- The process begins by querying the LLM with or without a few CoT examples for a set of training questions.
- The model generates k possible answers, introducing an element of uncertainty in its responses.
2. **Uncertainty Metric:**
- An uncertainty metric is calculated based on the disagreement among the k generated answers. This metric reflects the model's uncertainty about the most appropriate response.
3. **Selective Annotation:**
- The questions with the highest uncertainty, indicating a lack of consensus in the model's responses, are selected for annotation by humans.
- Humans provide new annotated exemplars specifically tailored to address the uncertainties identified by the LLM.
4. **Adaptive Learning:**
- The newly annotated exemplars are incorporated into the training data, enriching the model's understanding and adaptability for those specific questions.
- The model learns from the newly annotated examples, adjusting its responses based on the task-specific guidance provided.
:::
:::success
1. **主動提示(聚合)**

Active Prompting是透過動態選擇特定於任務的範例提示來增強LLMs對各種任務的適應性。思維鏈方法通常依賴一組固定的人工註解範例，這對於不同的任務可能並不總是最有效的。以下給出Active Prompting如何應對這項挑戰：
1. **動態查詢：**
- 流程首先對一組訓練問題來查詢LLMs，這可能帶有或不帶有少量CoT的範例。
- 模型產生k個可能的答案，在其響應中引入不確定性的元素。
2. **不確定性指標：**
- 不確定性的指標是基於k個生成的答案之間分歧的來計算。這個指標反映了模型對最合適的響應的不確定性。
3. **選擇性註解：**
- 不確定性最高的問題，指出模型的響應缺乏共識，再選出來由人類進行註解。
- 人類提供了新的帶註解的範例，專門用來解決LLMs所識別出來的不確定性。
4. **適應性學習：**
- 新註解的範例被納入訓練資料中，豐富了模型對這些特定問題的理解和適應性。
- 模型從新註解的範例中學習，根據提供的特定任務指導調整其響應。
:::

:::info
Active Prompting's dynamic adaptation mechanism enables LLMs to actively seek and incorporate task-specific examples that align with the challenges posed by different tasks. By leveraging human-annotated exemplars for uncertain cases, this approach contributes to a more contextually aware and effective performance across diverse tasks.
:::
:::success
Active Prompting的動態適應機制讓LLMs能夠主動尋求並整合與不同任務所帶來的挑戰相符的特定任務範例。透過利用人工註解的範例來處理不確定的情況，這種方法有助於在不同的任務中實現更具有contextually aware(情境感知)和更有效的表現。
:::
Image Source: Diao et al., (2023)
:::info
2. **Automatic Multi-step Reasoning and Tool-use (ART) (External Tools)**

ART emphasizes on task handling with LLMs. This framework integrates Chain-of-Thought prompting and tool usage by employing a frozen LLM. Instead of manually crafting demonstrations, ART selects task-specific examples from a library and enables the model to automatically generate intermediate reasoning steps. During test time, it integrates external tools into the reasoning process, fostering zero-shot generalization for new tasks. ART is not only extensible, allowing for human updates to task and tool libraries, but also promotes adaptability and versatility in addressing a variety of tasks with LLMs.
:::
:::success
2. **自動多步驟推理與工具使用(ART)(外部工具)**

ART強調LLMs的任務處理。這個框架透過使用一種frozen LLM整合了思維鏈的提示與工具的使用。 ART並不是手動制作演示，而是從庫(library)中選擇特定於任務的範例，然後讓模型能夠自動生成中間推理步驟。在測試期間，它將外部工具整合到推理過程中，促進新任務的Zero-shot泛化。 ART 不僅是可擴展的，還顧及到人類更新任務與工具庫，而且還能提高LLMs在處理各種任務的適應性和多功能性。
:::
Image Source: Paranjape et al., (2023)
:::info
3. **Chain-of-Knowledge (CoK)**

This framework aims to bolster LLMs by dynamically integrating grounding information from diverse sources, fostering more factual rationales and mitigating the risk of hallucination during generation. CoK operates through three key stages: reasoning preparation, dynamic knowledge adapting, and answer consolidation. It starts by formulating initial rationales and answers while identifying relevant knowledge domains. Subsequently, it refines these rationales incrementally by adapting knowledge from the identified domains, ultimately providing a robust foundation for the final answer. The figure below illustrates a comparison with other methods, highlighting CoK's incorporation of heterogeneous sources for knowledge retrieval and dynamic knowledge adapting.
:::
:::success
3. **知識鏈(CoK)**

該框架旨在透過動態整合來自不同來源的基礎信息、培育更多事實依據並減輕生成過程中產生幻覺的風險來支持LLMs。CoK的運作經歷三個關鍵階段：推理準備、動態知識適應和答案的合併。它首先制定初步的基本原理與答案，同時辨識相關的知識領域。隨後，它透過從已識別領域中所適應的知識來逐步完善這些基本原理，最終為最終答案提供堅實的基礎。下圖描述了與其他方法的比較，突顯CoK在知識檢索和動態知識適應方面對異質來源的整合能力。
:::
Image Source: Li et al. 2024
:::info
## Risks
Prompting comes with various risks, and prompt hacking is a notable concern that exploits vulnerabilities in LLMs. The risks associated with prompting include:

1. **Prompt Injection:**
- *Risk:* Malicious actors can inject harmful or misleading content into prompts, leading LLMs to generate inappropriate, biased, or false outputs.
- *Context:* Untrusted text used in prompts can be manipulated to make the model say anything the attacker desires, compromising the integrity of generated content.
2. **Prompt Leaking:**
- *Risk:* Attackers may extract sensitive information from LLM responses, posing privacy and security concerns.
- *Context:* Changing the user_input to attempt to leak the prompt itself is a form of prompt leaking, potentially revealing internal information.
3. **Jailbreaking:**
- *Risk:* Jailbreaking allows users to bypass safety and moderation features, leading to the generation of controversial, harmful, or inappropriate responses.
- *Context:* Prompt hacking methodologies, such as pretending, can exploit the model's difficulty in rejecting harmful prompts, enabling users to ask any question they desire.
4. **Bias and Misinformation:**
- *Risk:* Prompts that introduce biased or misleading information can result in outputs that perpetuate or amplify existing biases and spread misinformation.
- *Context:* Crafted prompts can manipulate LLMs into producing biased or inaccurate responses, contributing to the reinforcement of societal biases.
5. **Security Concerns:**
- *Risk:* Prompt hacking poses a broader security threat, allowing attackers to compromise the integrity of LLM-generated content and potentially exploit models for malicious purposes.
- *Context:* Defensive measures, including prompt-based defenses and continuous monitoring, are essential to mitigate security risks associated with prompt hacking.
:::
:::success
提示會帶來各種風險，而提示駭客攻擊是利用LLMs漏洞的一個值得注意的問題。與提示相關的風險包括：

1. **Prompt Injection：**
- *風險：* 惡意行為者可以將有害或誤導性的內容注入提示中，導致LLMs生成不適當、偏見或虛假的結果。
- *背景：* 提示中使用的不受信任的文字可以被操縱，造成模型說出攻擊者想要的任何內容，從而破壞生成內容的完整性。
2. **Prompt Leaking：**
- *風險：* 攻擊者可能從LLM的響應中提取敏感信息，從而造成隱私和安全問題。
- *背景：* 更改user_input來嘗試洩漏提示本身是提示洩漏的一種形式，可能會洩露內部信息。
3. **Jailbreaking：**
- *風險：* 越獄允許使用者繞過安全和審核功能，從而導致產生有爭議的、有害的或不適當的響應。
- *背景：* 提示駭客方法，像是假裝，可以利用模型難以拒絕有害提示的困難，使用戶能夠提出他們想要的任何問題。
4. **Bias and Misinformation：**
- *風險：* 引入偏見或誤導性信息的提示可能會導致輸出延續或放大現有偏見並傳播錯誤訊息。
- *背景：* 精心設計的提示可以操縱LLMs產生有偏見或不準確的響應，進而助長社會偏見。
5. **Security Concerns：**
- *風險：* 提示駭客攻擊會帶來更廣泛的安全威脅，使攻擊者能夠破壞LLM生成內容的完整性，並可能利用模型進行惡意目的。
- **Context：** 防禦措施，包括基於提示的防禦和持續監控，對於減輕與提示駭客攻擊相關的安全風險至關重要。
:::

:::info
To address these risks, it is crucial to implement robust defensive strategies, conduct regular audits of model behavior, and stay vigilant against potential vulnerabilities introduced through prompts. Additionally, ongoing research and development are necessary to enhance the resilience of LLMs against prompt-based attacks and mitigate biases in generated content.
:::
:::success
為了處理這些風險，實施穩健的防禦策略、定期審核模型行為並對提示引入的潛在漏洞保持警惕至關重要的。此外，持續的研究和發展對於增強LLMs對抗基於提示的攻擊的彈性並減少生成內容中的偏差是必要的。
:::
## Popular Tools
:::info
Here is a collection of well-known tools for prompt engineering. While some function as end-to-end app development frameworks, others are tailored for prompt generation and maintenance or evaluation purposes. The listed tools are predominantly open source or free to use and have demonstrated good adaptability. It's important to note that there are additional tools available, although they might be less widely recognized or require payment.
:::
:::success
這邊列出用於提示工程的知名工具。雖然有些功能作為端到端應用程式開發框架，但其他框架是專門為了技示生成和維護或評估目的而量身定制的。列出的工具主要是開源或免費使用的，並且表現出良好的適應性。值得注意的是，還有其它可以使用的工具，儘管它們可能不是那麼廣為人知或需要付費。
:::

:::info
1. **[PromptAppGPT](https://github.com/mleoking/PromptAppGPT):**
- *Description:* A low-code prompt-based rapid app development framework.
- *Features:* Low-code prompt-based development, GPT text and DALLE image generation, online prompt editor/compiler/runner, automatic UI generation, support for plug-in extensions.
- *Objective:* Enables natural language app development based on GPT, lowering the barrier to GPT application development.
2. **[PromptBench](https://github.com/microsoft/promptbench):**
- *Description:* A PyTorch-based Python package for the evaluation of LLMs.
- *Features:* User-friendly APIs for quick model performance assessment, prompt engineering methods (Few-shot Chain-of-Thought, Emotion Prompt, Expert Prompting), evaluation of adversarial prompts, dynamic evaluation to mitigate potential test data contamination.
- *Objective:* Facilitates the evaluation and assessment of LLMs with various capabilities, including prompt engineering and adversarial prompt evaluation.
3. **[Prompt Engine](https://github.com/microsoft/prompt-engine):**
- *Description:* An NPM utility library for creating and maintaining prompts for LLMs.
- *Background:* Aims to simplify prompt engineering for LLMs like GPT-3 and Codex, providing utilities for crafting inputs that coax specific outputs from the models.
- *Objective:* Facilitates the creation and maintenance of prompts, codifying patterns and practices around prompt engineering.
4. **[Prompts AI](https://github.com/sevazhidkov/prompts-ai):**
- *Description:* An advanced GPT-3 playground with a focus on helping users discover GPT-3 capabilities and assisting developers in prompt engineering for specific use cases.
- *Goals:* Aid first-time GPT-3 users, experiment with prompt engineering, optimize the product for use cases like creative writing, classification, and chat bots.
5. **[OpenPrompt](https://github.com/thunlp/OpenPrompt):**
- *Description:* A library built upon PyTorch for prompt-learning, adapting LLMs to downstream NLP tasks.
- *Features:* Standard, flexible, and extensible framework for deploying prompt-learning pipelines, supporting loading PLMs from huggingface transformers.
- *Objective:* Provides a standardized approach to prompt-learning, making it easier to adapt PLMs for specific NLP tasks.
6. **[Promptify](https://github.com/promptslab/Promptify):**
- *Features:* Test suite for LLM prompts, perform NLP tasks in a few lines of code, handle out-of-bounds predictions, output provided as Python objects for easy parsing, support for custom examples and samples, run inference on models from the Huggingface Hub.
- *Objective:* Aims to facilitate prompt testing for LLMs, simplify NLP tasks, and optimize prompts to reduce OpenAI token costs.
:::
:::success
1. **[PromptAppGPT](https://github.com/mleoking/PromptAppGPT):**
- *描述：* 一個low-code prompt-based的快速應用程式開發框架。
- *特點：* low-code prompt-based的開發，GPT文本和DALLE圖像生成，線上提示編輯器/編譯器/執行器，自動UI生成，支援插件擴充。
- *目標：* 實現基於GPT的自然語言應用程式開發，降低GPT應用程式開發的門檻。
2. **[PromptBench](https://github.com/microsoft/promptbench):**
- *描述：* 基於PyTorch的Python套件，用於LLMs的評估。
- *特點：* 使用者友好的API，用於快速模型性能評估、提示工程方法(Few-shot Chain-of-Thought, Emotion Prompt, Expert Prompting)、對抗性提示評估、動態評估以減輕潛在的測試數據污染。
- *目標：* 促進具有各種能力的LLMs的評估和評估，包括提示工程和對抗性提示評估。
3. **[提示引擎](https://github.com/microsoft/prompt-engine):**
- *描述：* 用於建立和維護LLM提示的NPM工具庫。
- *背景：* 旨在簡化像是GPT-3和Codex等LLMs的提示工程，提供用於制作輸入的工具，以誘導模型的特定輸出。
- *目標：* 促進提示的建立和維護，圍繞在提示工程的編寫模式與實踐。
4. **[提示AI](https://github.com/sevazhidkov/prompts-ai):**
- *描述：* 一個先進的GPT-3遊樂場，專注於幫助使用者發現GPT-3的功能並協助開發人員針對特定用例進行提示工程。
- *目標：* 目標是幫助首次使用GPT-3的使用者嘗試提示工程來最佳化產品像是創意寫作、分類和聊天機器人等用例。
5. **[OpenPrompt](https://github.com/thunlp/OpenPrompt):**
- *描述：* 基於PyTorch建構的函式庫，用於prompt-learnin，使LLM適應下游NLP任務。
- *特點：* 標準、靈活且可擴展的框架，用於部署prompt-learnin pipelines，支援從 Huggingface transformers載入PLMs。
- *目標：* 提供標準化的prompt-learnin方法，使PLMs更容易適應特定的NLP任務。
6. **[提示](https://github.com/promptslab/Promptify):**
- *功能：* LLM提示的測試套件，用少少幾行的程式碼執行NLP任務，處理out-of-bounds prediction(超出預期的預測？)，以Python物件形式提供輸出以便於解析，支援自訂範例和樣本，執行來自Huggingface Hub的模型上的推理。
- *目標：* 旨在促進LLMs的提示測試、簡化NLP任務並最佳化提示以降低OpenAI token costs。
:::