Try   HackMD

AI / ML領域相關學習筆記入口頁面

Deeplearning.ai GenAI/LLM系列課程筆記

GenAI
RAG
AI Agents

原理可參考這篇綜論: How Agents for LLM Perform Task Planning。大型語言模型的代理如何進行任務規劃

相關framework選擇

AI Agents in LangGraph

Build an Agent from Scratch。 從零開始建立AI代理

REACT(偕同的理解+行動)實作

  • REACT: SYNERGIZING REASONING AND ACTING IN LANGUAGE MODELS

  • Prompt Engineering Guide。ReAct Prompting

    Image Not Showing Possible Reasons
    • The image was uploaded to a note which you don't have access to
    • The note which the image was originally uploaded to has been deleted
    Learn More →

  • ReAct 框架概念主要包含下列三個元素：

    • Reasoning Traces（推理軌跡）: 模型在處理任務時所經歷的內部思考過程和步驟
    • Actions（行動）: 模型與外部環境互動的步驟，如檢索信息或執行特定任務
    • Observations（觀察）: 執行行動後從環境中獲得的回饋或資料

    Chain-of-thought（CoT）提示展示了 LLM 執行推理軌跡以生成涉及算術和常識推理等任務的回答的能力。但其缺乏訪問外部世界的能力或無法更新知識，會導致事實幻覺和錯誤傳播等問題。
    ReAct 是一種將推理和行動結合起來的通用範式。ReAct 提示 LLM 生成任務的口頭推理軌跡和行動。這使系統能夠執行動態推理，以創建、維護和調整行動計劃，同時還能與外部環境（如維基百科）進行互動，將額外的信息納入推理過程。

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

  • 分析
    • 推理過程（Thoughts）：每一步的思考都反映了如何逐步接近答案，首先是了解Apple Remote的設計初衷，接著找出相關的控制軟體，再尋找這些軟體的具體控制設備。
    • 行為（Acts）：每一步的行為是根據思考所進行的具體操作，這些操作包括在知識庫中搜索相關信息。
    • 觀察（Observations）：每一步行為後的觀察記錄了從環境（知識庫）中獲得的反饋信息，這些信息進一步指引了下一步的思考和行為。
  • 結果
    • 最終的答案是鍵盤功能鍵（keyboard function keys），因為經過多步推理和行為，發現Front Row（軟體）可以由Apple Remote或鍵盤功能鍵控制。這張圖展示了ReAct方法如何有效結合推理和行為來解決問題，並找到正確答案。

code

Image Not Showing Possible Reasons
  • The image was uploaded to a note which you don't have access to
  • The note which the image was originally uploaded to has been deleted
Learn More →

  • 簡單的class Agent 實作

    • 在def __call__中，
      • 每次 Agent得到使用者的message後
      • 就調用self.execute()，調用llm完成一次推理（Thoughts）/行為（Acts）動，返回對話結果/觀察（Observations）
      • 再將上次的結果/觀察（Observations）作為下次丟給Agent的訊息
    



















    ​​​​class Agent:
​​​​    def __init__(self, system=""):
​​​​        self.system = system
​​​​        self.messages = []
​​​​        if self.system:
​​​​            self.messages.append({"role": "system", "content": system})

​​​​    def __call__(self, message):
​​​​        self.messages.append({"role": "user", "content": message})
​​​​        result = self.execute()
​​​​        self.messages.append({"role": "assistant", "content": result})
​​​​        return result

​​​​    def execute(self):
​​​​        completion = client.chat.completions.create(
​​​​                        # model="gpt-35-turb 
​​​​                        temperature=0,
​​​​                        messages=self.messages)
​​​​        return completion.choices[0].message.content

  • prompt

    • 告訴LLM在 Thought, Action, PAUSE, Observation的循環中
    • 在loop結果中要輸出: Answer
    • 使用Thought描述對問題的看法
    • 使用Action執行可用的操作，然後返回 PAUSE
    • Observation即是執行上述行動的結果
    • :pencil2: 注意這邊的狀態是用字首大寫
    































    ​​​​    prompt = """
​​​​    You run in a loop of Thought, Action, PAUSE, Observation.
​​​​    At the end of the loop you output an Answer
​​​​    Use Thought to describe your thoughts about the question you have been asked.
​​​​    Use Action to run one of the actions available to you - then return PAUSE.
​​​​    Observation will be the result of running those actions.

​​​​    Your available actions are:

​​​​    calculate:
​​​​    e.g. calculate: 4 * 7 / 3
​​​​    Runs a calculation and returns the number - uses Python so be sure to use floating point syntax if necessary

​​​​    average_dog_weight:
​​​​    e.g. average_dog_weight: Collie
​​​​    returns average weight of a dog when given the breed

​​​​    Example session:

​​​​    Question: How much does a Bulldog weigh?
​​​​    Thought: I should look the dogs weight using average_dog_weight
​​​​    Action: average_dog_weight: Bulldog
​​​​    PAUSE

​​​​    You will be called again with this:

​​​​    Observation: A Bulldog weights 51 lbs

​​​​    You then output:

​​​​    Answer: A bulldog weights 51 lbs
​​​​    """.strip()
    • 給定的行動方案/工具選擇
    















    ​​​​def calculate(what):
​​​​    return eval(what)

​​​​def average_dog_weight(name):
​​​​    if name in "Scottish Terrier": 
​​​​        return("Scottish Terriers average 20 lbs")
​​​​    elif name in "Border Collie":
​​​​        return("a Border Collies average weight is 37 lbs")
​​​​    elif name in "Toy Poodle":
​​​​        return("a toy poodles average weight is 7 lbs")
​​​​    else:
​​​​        return("An average dog weights 50 lbs")

​​​​known_actions = {
​​​​    "calculate": calculate,
​​​​    "average_dog_weight": average_dog_weight

    • 建立任務循環loop

      • 從回應結果中抽取出有關action的部分
        • actions = [action_re.match(a) for a in result.split('\n') if action_re.match(a)]
      • 觀察動作後的狀態
        • observation = known_actions[action](action_input)
      


























      
​​​​​​​​action_re = re.compile('^Action: (\w+): (.*)$')   # python regular expression to selection action
​​​​
​​​​​​​​def query(question, max_turns=5):
​​​​​​​​    i = 0
​​​​​​​​    bot = Agent(prompt)
​​​​​​​​    next_prompt = question
​​​​​​​​    while i < max_turns:
​​​​​​​​        i += 1
​​​​​​​​        result = bot(next_prompt)
​​​​​​​​        print(result)
​​​​​​​​        actions = [
​​​​​​​​            action_re.match(a) 
​​​​​​​​            for a in result.split('\n') 
​​​​​​​​            if action_re.match(a)
​​​​​​​​        ]
​​​​​​​​        if actions:
​​​​​​​​            # There is an action to run
​​​​​​​​            action, action_input = actions[0].groups()
​​​​​​​​            if action not in known_actions:
​​​​​​​​                raise Exception("Unknown action: {}: {}".format(action, action_input))
​​​​​​​​            print(" -- running {} {}".format(action, action_input))
​​​​​​​​            observation = known_actions[action](action_input)
​​​​​​​​            print("Observation:", observation)
​​​​​​​​            next_prompt = "Observation: {}".format(observation)
​​​​​​​​        else:
​​​​​​​​            return

      • 測試提問結果

        • 第一輪Thought會先分析問題、決定後面採取的動作
        • 接下來幾輪會重複執行Thought-Action-PAUSE-Observation-Thought，直到獲得答案
          • 某種角度來看，是在LLM內部的參數空間及透過MESSAGE容器儲存檢索/生成結果來避免幻覺
        


        ​​​​​​​​​​​​question = """I have 2 dogs, a border collie and a scottish terrier. \
​​​​​​​​​​​​What is their combined weight"""
​​​​​​​​​​​​query(question)

        ​​​​​​​​​​​​# Thought: To find the combined weight of the two dogs, I need to use the average_dog_weight action for each breed and then add the results together.

​​​​​​​​​​​​# Action: average_dog_weight: Border Collie
​​​​​​​​​​​​# PAUSE
​​​​​​​​​​​​#  -- running average_dog_weight Border Collie
​​​​​​​​​​​​# Observation: a Border Collies average weight is 37 lbs
​​​​​​​​​​​​# Thought: Now that I have the average weight of a Border Collie, I can use the average_dog_weight action for the Scottish Terrier.

​​​​​​​​​​​​# Action: average_dog_weight: Scottish Terrier
​​​​​​​​​​​​# PAUSE
​​​​​​​​​​​​#  -- running average_dog_weight Scottish Terrier
​​​​​​​​​​​​# Observation: Scottish Terriers average 20 lbs
​​​​​​​​​​​​# Thought: Now that I have the average weight of both the Border Collie and the Scottish Terrier, I can calculate their combined weight by adding the two values together.

​​​​​​​​​​​​# Action: calculate: 37 + 20
​​​​​​​​​​​​# PAUSE
​​​​​​​​​​​​#  -- running calculate 37 + 20
​​​​​​​​​​​​# Observation: 57
​​​​​​​​​​​​# Answer: The combined weight of a Border Collie and a Scottish Terrier is 57 lbs.
Last changed by 
 
YH Hsu
www.linkedin.com/in/yung-hui-hsu-226b50a6
0
3072

Read more

[GenAI][AI Agents] Long-Term Agentic Memory With LangGraph - Introduction to Agent Memory

Long-Term Agentic Memory With LangGraph

Apr 14, 2025
[GenAI][AI Agents] Long-Term Agentic Memory With LangGraph - Baseline Email Assistant

Large Language Models with Semantic Search。大型語言模型與語義搜索

Apr 1, 2025
[AI Agents in LangGraph](https://learn.deeplearning.ai/courses/ai-agents-in-langgraph/lesson/1/introduction)

LanaGraph Components

Mar 19, 2025
AI / ML領域相關學習筆記入口頁面

Mar 19, 2025
Read more from YH Hsu
Published on HackMD