tags:`AFS`

Function Calling

Introduction

在 API 呼叫中，您可以描述多個函式讓模型選擇，並輸出包含選中的函數名稱及參數的 JSON Object，讓應用或代理人程式調用模型選擇的函式。Chat Completions API 不會調用該函式而是生成 JSON 讓您可用於在代碼中調用函式。

使用方式

開發者提供函式列表並對大語言模型輸入問題。
開發者解析大語言模型輸出的結構化資料，取得函式與對應的參數後，讓應用或代理人程式呼叫 API 並獲得回傳的結果。
將 API 回傳的結果放到對話內容並傳給大語言模型做總結。

使用範例

Prompt Format for Function Calling












<s>[INST] <<FUNC>>
{{ function定義 }}
<</FUNC>>

<<SYS>>
{{ system_prompt }}
<</SYS>>

{{ user_message }} [/INST] <|func_start|> {{ function call 的內容}} <|func_end|> </s><s><<RET>> name="This_is_function_name"
{{ function 回傳的結果 }}
<</RET>>

Chat Completions API

給定包含對話的訊息列表，模型將回傳生成訊息或呼叫函式。

開發者提供函式列表並對大語言模型輸入問題

Field	Type	Require	Description
functions	array	Optional	A list of functions the model may generate JSON inputs for.

Example of RESTful HTTP Request











































export API_KEY={API_KEY}
export API_URL=https://{DOMAIN_URL}/text-generation

curl -X POST "${API_URL}/api/models/conversation" \
-H "accept: application/json" \
-H "X-API-KEY:${API_KEY}" \
-H "content-type: application/json" \
-d '{
  "model": "Llama-2-7b-chat-hf",
  "messages": [
    {
      "role": "user",
      "content": "What is the weather like in Boston?"
    }],
  "functions": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city and state, e.g. San Francisco, CA"
          },
          "unit": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"]
          }
        },
        "required": ["location"]
      }
    }],
  "parameters": {
      "show_probabilities": false,
      "max_new_tokens": 50,
      "frequence_penalty": 1,
      "temperature": 0.5,
      "top_k": 100,
      "top_p": 0.93
  },
  "stream": false
}'

Use prompt


























































<s>[INST] <<FUNC>>
{
    "name": "get_current_weather", 
    "description": "Get the current weather", 
    "parameters": {
        "type": "object", 
        "properties": {
            "location": {
                "type": "string", 
                "description": "The city and state, e.g. San Francisco, CA"
            }
        }, 
        "required": ["location"]
    }
},
{
    "name": "func_one",
    "description": "function 1", 
    "parameters": {
        "properties": {
            "para1": {
                "description": "parameter 1", 
                "type": "string"
            }, 
            "para2": {
                "description": "parameter 2", 
                "type": "integer"
            }
        }, 
        "required": ["para1"], 
        "type": "object"
    }
}
{
    "name": "func_two",
    "description": "function 2", 
    "parameters": {
        "properties": {
            "para1": {
                "description": "parameter 1", 
                "type": "string"
            }, 
            "para2": {
                "description": "parameter 2", 
                "type": "integer"
            }
        }, 
        "required": ["para1"], 
        "type": "object"
    }
}
<</FUNC>>

<<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>

What is the weather like in New York, NY? [/INST]

將 prompt 輸入模型後得到函式呼叫的結果

開發者解析大語言模型輸出的結構化資料，取得函式與對應的參數後，呼叫 API 並獲得回傳的結果

Field	Type	Require	Description
function_call	string or object	Optional	JSON format that adheres to the function signature

Example of RESTful HTTP Response















{
  "generated_text": "",
  "function_call": {
    "name": "get_current_weather",
    "arguments": {
      "location": "Boston"
    }
  },
  "details":null,
  "total_time_taken": "1.18 sec",
  "prompt_tokens": 181,
  "generated_tokens": 45,
  "total_tokens": 226,
  "finish_reason": "function_call"
}

將 API 回傳的結果放到對話內容並傳給大語言模型做總結

Field	value
role	function
name	The function name to call
content	The response message from the API

Example of RESTful HTTP Request











































export API_KEY={API_KEY}
export API_URL=https://{DOMAIN_URL}/text-generation

curl -X POST "${API_URL}/api/models/conversation" \
-H "accept: application/json" \
-H "X-API-KEY:${API_KEY}" \
-H "content-type: application/json" \
-d '{
  "model": "Llama-2-7b-chat-hf",
  "messages": [
    {"role": "user", "content": "What is the weather like in Boston?"},
    {"role": "assistant", "content": null, "function_call": {"name": "get_current_weather", "arguments": {"location": "Boston, MA"}}},
    {"role": "function", "name": "get_current_weather", "content": "{\"temperature\": \"22\", \"unit\": \"celsius\", \"description\": \"Sunny\"}"}
  ],
  "functions": [
    {
      "name": "get_current_weather",
      "description": "Get the current weather in a given location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city and state, e.g. San Francisco, CA"
          },
          "unit": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"]
          }
        },
        "required": ["location"]
      }
    }],
  "parameters": {
      "show_probabilities": false,
      "max_new_tokens": 50,
      "frequence_penalty": 1,
      "temperature": 0.5,
      "top_k": 100,
      "top_p": 0.93
  },
  "stream": false
}'

Example of RESTful HTTP Response

{
"generated_text":" The current weather in Boston is sunny with a temperature of 22 degrees Celsius. ",
"details":null,
"total_time_taken":"0.64 sec",
"prompt_tokens":230,
"generated_tokens":23,
"total_tokens":253,
"finish_reason":"eos_token"
}

Use prompt




























































<s>[INST] <<FUNC>>
{
    "name": "get_current_weather", 
    "description": "Get the current weather", 
    "parameters": {
        "type": "object", 
        "properties": {
            "location": {
                "type": "string", 
                "description": "The city and state, e.g. San Francisco, CA"
            }
        }, 
        "required": ["location"]
    }
},
{
    "name": "func_one",
    "description": "function 1", 
    "parameters": {
        "properties": {
            "para1": {
                "description": "parameter 1", 
                "type": "string"
            }, 
            "para2": {
                "description": "parameter 2", 
                "type": "integer"
            }
        }, 
        "required": ["para1"], 
        "type": "object"
    }
},
{
    "name": "func_two",
    "description": "function 2", 
    "parameters": {
        "properties": {
            "para1": {
                "description": "parameter 1", 
                "type": "string"
            }, 
            "para2": {
                "description": "parameter 2", 
                "type": "integer"
            }
        }, 
        "required": ["para1"], 
        "type": "object"
    }
}
<</FUNC>>

<<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>

What is the weather like in New York, NY? [/INST] <|func_start|> {"name":"get_current_weather","arguments":{"location":"New York, NY"}} <|func_end|> </s><s><<RET>> name="get_current_weather"
{"Temperature": "57F", "Condition": "Raining"}
<</RET>>

將 prompt 輸入模型後得到總結:

tags:AFS

Function Calling

Function Calling

Introduction

使用方式

使用範例

Prompt Format for Function Calling

Chat Completions API

Read more

AFS API 說明文件

AFS MyCoder

How to Use Tool Call of FFM Conversation API with Llama3 Series

AFS Coder - GitCoder

tags:`AFS`