###### tags:`AFS`
# Function Calling
## Function Calling
### Introduction
在 API 呼叫中，您可以描述多個函式讓模型選擇，並輸出包含選中的函數名稱及參數的 JSON Object，讓應用或代理人程式調用模型選擇的函式。Chat Completions API 不會調用該函式而是生成 JSON 讓您可用於在代碼中調用函式。
### 使用方式
1. 開發者提供函式列表並對大語言模型輸入問題。
2. 開發者解析大語言模型輸出的結構化資料，取得函式與對應的參數後，讓應用或代理人程式呼叫 API 並獲得回傳的結果。
3. 將 API 回傳的結果放到對話內容並傳給大語言模型做總結。
### 使用範例
### Prompt Format for Function Calling
```python=
<s>[INST] <<FUNC>>
{{ function定義 }}
<</FUNC>>
<<SYS>>
{{ system_prompt }}
<</SYS>>
{{ user_message }} [/INST] <|func_start|> {{ function call 的內容}} <|func_end|> </s><s><<RET>> name="This_is_function_name"
{{ function 回傳的結果 }}
<</RET>>
```
### Chat Completions API
給定包含對話的訊息列表，模型將回傳生成訊息或呼叫函式。
1. 開發者提供函式列表並對大語言模型輸入問題
| Field | Type | Require | Description |
| -------- | -------- | -------- | -------- |
| **functions** | array | Optional | A list of functions the model may generate JSON inputs for.|
* Example of RESTful HTTP Request
```python=
export API_KEY={API_KEY}
export API_URL=https://{DOMAIN_URL}/text-generation
curl -X POST "${API_URL}/api/models/conversation" \
-H "accept: application/json" \
-H "X-API-KEY:${API_KEY}" \
-H "content-type: application/json" \
-d '{
"model": "Llama-2-7b-chat-hf",
"messages": [
{
"role": "user",
"content": "What is the weather like in Boston?"
}],
"functions": [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
}],
"parameters": {
"show_probabilities": false,
"max_new_tokens": 50,
"frequence_penalty": 1,
"temperature": 0.5,
"top_k": 100,
"top_p": 0.93
},
"stream": false
}'
```
* Use prompt
```python=
<s>[INST] <<FUNC>>
{
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
},
{
"name": "func_one",
"description": "function 1",
"parameters": {
"properties": {
"para1": {
"description": "parameter 1",
"type": "string"
},
"para2": {
"description": "parameter 2",
"type": "integer"
}
},
"required": ["para1"],
"type": "object"
}
}
{
"name": "func_two",
"description": "function 2",
"parameters": {
"properties": {
"para1": {
"description": "parameter 1",
"type": "string"
},
"para2": {
"description": "parameter 2",
"type": "integer"
}
},
"required": ["para1"],
"type": "object"
}
}
<</FUNC>>
<<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>
What is the weather like in New York, NY? [/INST]
```
> 將 prompt 輸入模型後得到函式呼叫的結果
>
> 
2. 開發者解析大語言模型輸出的結構化資料，取得函式與對應的參數後，呼叫 API 並獲得回傳的結果
| Field | Type | Require | Description |
| -------- | -------- | -------- | -------- |
|**function_call** | string or object | Optional | JSON format that adheres to the function signature |
* Example of RESTful HTTP Response
```python=
{
"generated_text": "",
"function_call": {
"name": "get_current_weather",
"arguments": {
"location": "Boston"
}
},
"details":null,
"total_time_taken": "1.18 sec",
"prompt_tokens": 181,
"generated_tokens": 45,
"total_tokens": 226,
"finish_reason": "function_call"
}
```
3. 將 API 回傳的結果放到對話內容並傳給大語言模型做總結
| Field |value |
| -------- | -------- |
|**role** | ***function*** |
|**name** | The function name to call |
|**content** | The response message from the API |
* Example of RESTful HTTP Request
```python=
export API_KEY={API_KEY}
export API_URL=https://{DOMAIN_URL}/text-generation
curl -X POST "${API_URL}/api/models/conversation" \
-H "accept: application/json" \
-H "X-API-KEY:${API_KEY}" \
-H "content-type: application/json" \
-d '{
"model": "Llama-2-7b-chat-hf",
"messages": [
{"role": "user", "content": "What is the weather like in Boston?"},
{"role": "assistant", "content": null, "function_call": {"name": "get_current_weather", "arguments": {"location": "Boston, MA"}}},
{"role": "function", "name": "get_current_weather", "content": "{\"temperature\": \"22\", \"unit\": \"celsius\", \"description\": \"Sunny\"}"}
],
"functions": [
{
"name": "get_current_weather",
"description": "Get the current weather in a given location",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
},
"unit": {
"type": "string",
"enum": ["celsius", "fahrenheit"]
}
},
"required": ["location"]
}
}],
"parameters": {
"show_probabilities": false,
"max_new_tokens": 50,
"frequence_penalty": 1,
"temperature": 0.5,
"top_k": 100,
"top_p": 0.93
},
"stream": false
}'
```
* Example of RESTful HTTP Response
> {
> "generated_text":" The current weather in Boston is sunny with a temperature of 22 degrees Celsius. ",
> "details":null,
> "total_time_taken":"0.64 sec",
> "prompt_tokens":230,
> "generated_tokens":23,
> "total_tokens":253,
> "finish_reason":"eos_token"
> }
* Use prompt
```python=
<s>[INST] <<FUNC>>
{
"name": "get_current_weather",
"description": "Get the current weather",
"parameters": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA"
}
},
"required": ["location"]
}
},
{
"name": "func_one",
"description": "function 1",
"parameters": {
"properties": {
"para1": {
"description": "parameter 1",
"type": "string"
},
"para2": {
"description": "parameter 2",
"type": "integer"
}
},
"required": ["para1"],
"type": "object"
}
},
{
"name": "func_two",
"description": "function 2",
"parameters": {
"properties": {
"para1": {
"description": "parameter 1",
"type": "string"
},
"para2": {
"description": "parameter 2",
"type": "integer"
}
},
"required": ["para1"],
"type": "object"
}
}
<</FUNC>>
<<SYS>>
You are a helpful, respectful and honest assistant.
<</SYS>>
What is the weather like in New York, NY? [/INST] <|func_start|> {"name":"get_current_weather","arguments":{"location":"New York, NY"}} <|func_end|> </s><s><<RET>> name="get_current_weather"
{"Temperature": "57F", "Condition": "Raining"}
<</RET>>
```
> 將 prompt 輸入模型後得到總結:
> 
Sign in with Wallet
Connect another wallet