# Creating an Agent with Semantic Kernel

This guide focuses on using **Microsoft Semantic Kernel** combined with the **LINE platform** to implement a highly extensible and flexible AI Agent system. It adopts a **single main Agent with multiple Plugin tools**, supporting multi-intent task handling. The main example in this article is the **MarketAgent** and its associated plugins: **SearchPlugin**, **AnalysisPlugin**, and **SuggestionPlugin**.

> Note: This document focuses on Agent and Plugin implementation details. For ChatHistory memory management and RBAC access control, please refer to their respective technical documents.

---

## System Design: Single Agent with Multiple Tools vs. Mini-Agent

### Limitations of Multi-Agent Architecture

Initially, I experimented with a **multi-agent architecture**, using a central **TriageAgent** responsible for routing queries to specific mini-agents (e.g., QueryAgent, SuggestionAgent, AnalysisAgent). Each mini-agent was bound to its own plugin tool.

Original Mini-Agent architecture diagram:

While this design seemed clearly divided, it introduced significant limitations when handling complex inputs:

1. **Only one agent could be assigned at a time**, preventing proper handling of multi-intent queries.
2. **TriageAgent limitations**: Since its role was to route queries to a single Agent, the system lost flexibility and autonomy.

Example of a multi-intent problem:

> User asks: "Which products are good for promotion this week? And do these products still have enough stock?"

This is a **query + suggestion** multi-intent input. The original architecture would route it to SuggestionAgent, missing the stock query and resulting in an incomplete answer.

Lack of flexibility and autonomy:

> User asks: "What can you do? What features do you have?"

This is a general query, but since TriageAgent must choose a single Mini-Agent, the response loses autonomy and accuracy.

---

### Single Main Agent + Multiple Plugin Tools

I restructured the system into a **single ChatCompletionAgent with multiple Plugin Tools**, where the main agent manages all tools and task routing.

Advantages of this design:

* **Flexible multi-plugin invocation**: Handle complex multi-intent queries (e.g., query + analysis).
* **Scalable**: Add new features by plugging in a new plugin, no need to add new agents.
* **Supports function calling auto-planning**: The agent can autonomously select the right tool(s) to answer based on user input.

This plugin-centric design removes the single-task limitation, enabling the assistant to flexibly handle compound tasks.

---

## Semantic Kernel: Agents and Plugins

The Semantic Kernel Plugin system and function calling are the key technologies that enable the “single agent with multiple tools” design. When a set of tools is registered as a plugin with clear descriptions and parameter schemas, the ChatCompletionAgent can automatically:

1. Determine whether external tools are needed.
2. Select the appropriate plugin.
3. Call the plugin and obtain results.
4. Chain multiple function calls if needed.
5. Combine results and generate the final answer.

Official flow diagram:

* Step 2: Send user input to the model
* Step 3: Model analyzes and decides if a tool is needed
* Step 4: If tool response is sufficient, return answer directly
* Step 5: If insufficient, invoke next tool(s)
* Step 6: Integrate results and repeat Step 3

The major advantage: **tool selection, invocation, and result integration are automated. Developers only need to clearly define plugin functions and parameters.**

All tool usage history is also included in the Chat History, giving the model memory of previous actions and results, thus improving decision continuity and accuracy.

### Agent and Plugin Limitations

OpenAI also provides practical limits and recommendations:

In short: **the more tools you add, the higher the chance of tool-selection errors.**

---

## Implementation Example

### Agent Setup

Example of setting up an AI Agent in Semantic Kernel, using `MarketAgent`:

from semantic_kernel.agents import ChatCompletionAgent
from semantic_kernel.connectors.ai.open_ai import OpenAIChatCompletion
from semantic_kernel.connectors.ai.prompt_execution_settings import PromptExecutionSettings
from semantic_kernel.functions.kernel_arguments import KernelArguments

prompt_settings = PromptExecutionSettings(
        temperature=0.2,         # Lower = less creative
        top_p=0.1,               # Controls nucleus sampling
        frequency_penalty=0.0,   # Repeat discouragement
        presence_penalty=0.0,    # New topic encouragement
    )

market_agent = ChatCompletionAgent(
    service=service,
    name="MarketAgent",
    arguments = KernelArguments(settings=prompt_settings),
    instructions= MARKET_AGENT_PROMPT,
    plugins = [search_plugin, analysis_plugin, suggestion_plugin]
)

---

### Plugin Setup

Each Plugin should follow this standard structure:

from semantic_kernel.functions import kernel_function
from typing import Annotated
from dataclasses import dataclass

@dataclass
class PluginResponse:
    success: bool
    data: dict
    message: str
    error: str = None

class SearchPlugin:

    @kernel_function(
        name="search_function",
        description="Product search function"
    )
    def search_function(self, param: Annotated[str, "Parameter description"]) -> PluginResponse:
        try:
            # Product search logic
            result = {"products": []}
            return PluginResponse(success=True, data=result, message="Search successful")
        except Exception as e:
            return PluginResponse(success=False, data={}, message="Search failed", error=str(e))

#### Plugin Best Practices

1. **Single Responsibility**: Each plugin focuses on a single function.
2. **Unified Response Format**: Use a standardized PluginResponse.
3. **Clear Descriptions**: Provide detailed function descriptions.
4. **Parameter Annotations**: Use Annotated for parameter explanations.
5. **Robust Error Handling**: Capture and report errors clearly.

---

### Invoking the Agent

After defining the Agent and Plugins, here’s an example of calling MarketAgent to process user messages:

thread = thread_manager.get_thread("uuid")

market_agent_route = await market_agent.get_response(
        messages = question,
        thread = thread
    )

> In this example, `thread` is provided by Semantic Kernel’s built-in classes, usually used with ChatHistory(), ensuring independent and consistent conversation context for each user.

---

## Extended Features

After building the base Agent and Plugins, engineers may want to enhance system maintainability, scalability, and security. Two key topics here are **memory management (ChatHistory)** and **role-based access control (RBAC)**.

### ChatHistory Management: Redis vs. ChatHistoryAgentThread

Two common approaches to agent memory management:

* **Redis**:

* Pros: Simple and quick to implement, suitable for basic text-based context storage.
* Cons: Cannot store structured dialogue or tool-call details. Each round requires re-parsing context → lower efficiency and higher error rate.

* **ChatHistoryAgentThread**:

* Pros: Stores complete dialogues, plugin calls, and results. Maintains state and parameters, reduces redundant computation, and improves consistency and accuracy.
* Cons: Requires additional configuration and management, more complex architecture.

| Metric | Redis | ChatHistoryAgentThread |
| ------------------- | ------------------- | ----------------------- |
| Context retention | Text only | Full structured context |
| Tool-call tracking | No | Yes |
| Parameter retention | Re-parsed each turn | Maintained |
| Token usage | High (re-parsed) | Low (stateful) |
| Error rate | High | Low |
| Debugging ability | Limited | Full traceability |

> For detailed implementation, see [ChatHistory Management Technical Doc](https://hackmd.io/@WoodyChang1121/BJoXWPmHgl).

### RBAC Access Control

RBAC (Role-Based Access Control) restricts access based on user role and integrates email verification for security.

* **Authentication**: Email verification codes with LINE ID binding.
* **Role Definition**: Map user roles to permissions via `function_role_access_map`.
* **Scalability**: Easily add roles and features as system grows.

> For detailed implementation, see [RBAC Technical Doc](https://hackmd.io/@WoodyChang1121/ryfZCX5Heg).

---

## Conclusion

By adopting Semantic Kernel’s **single agent + multi-plugin** architecture, engineers can build AI systems that are highly flexible, extensible, and maintainable. Using **MarketAgent** as an example, we demonstrated practical implementation details and introduced advanced topics such as **ChatHistory memory management** and **RBAC access control**. This approach enables rapid expansion of features, efficient debugging, and creation of more intelligent, integrated AI services.