# Microsfot Foundry IQ Workshop ![image](https://hackmd.io/_uploads/B19jVnumbl.png) ## 一. Azure 資源準備 1.**Required Permission for Subscription** (擇一即可): + Resource Group Owner: 擁有管理指定資源群組的完整權限。 + Contributor: 可在訂閱或資源群組中建立及管理所有類型的 Azure 資源,但無法授權其他使用者。 2.**Require Access for GPT5.2:** + 一個Subscription Id 對應一個申請單 [Request Form](https://customervoice.microsoft.com/Pages/ResponsePage.aspx?id=v4j5cvGGr0GRqy180BHbR7en2Ais5pxKtso_Pz4b1_xUQ1VGQUEzRlBIMVU2UFlHSFpSNkpOR0paRSQlQCN0PWcu) 3.**建立Microsoft AI Foundry** + Region: **East US2** + 🔗 參考資料: [Create a project for Microsoft Foundry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/create-projects?view=foundry&tabs=foundry) [Foundry Models sold directly by Azure](https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure?view=foundry-classic&tabs=global-standard-aoai%2Cstandard-chat-completions%2Cglobal-standard&pivots=azure-openai) [Supported models in Foundry Agent Service](https://learn.microsoft.com/en-us/azure/ai-foundry/agents/concepts/model-region-support?view=foundry-classic&tabs=global-standard) 2 **AI Foundry 模型部署** + 💬 Chat Completion 模型 - `gpt-5.2` - `gpt-4.1-mini` + 🔍 Embedding 模型 - `text-embedding-3-large` + 🔗 參考資料: [Microsoft Foundry Quickstart](https://learn.microsoft.com/en-us/azure/ai-foundry/quickstarts/get-started-code?view=foundry&tabs=python) 3 **部署Azure AI Search** + Region: 請選擇有support `Semantic rank` 的region + Pricing tier: `Standard` 💡 **費用提示**:Azure AI Search 若使用 **Standard** Pricing Tier,**每月約需 USD \$300 左右**。 > 若是課程或團隊練習,**建議學員共用同一組 Azure AI Search 服務**,可有效降低成本。 + 🔗 參考資料: [Create an Azure AI Search service in the Azure portal](https://learn.microsoft.com/en-us/azure/search/search-create-service-portal) 4 **部署Azure Storage Account** + 🔗 參考資料: [Create an Azure storage account](https://learn.microsoft.com/en-us/azure/storage/common/storage-account-create?tabs=azure-portal) + 建立Container: - `evaluation`: 請上傳考核、請假、獎懲 資料夾的檔案 - `bonus`: 待遇、獎金、 酬勞 資料夾的檔案 ## 二、本地Python環境準備 ### 1. 己完成下列本機環境安裝 #### Python環境 + 本機Python環境 3.10以上版本 + **驗證安裝**: ```shell python ``` #### Vistual Studio Code環境 + 本機Vistual Studio Code + 安裝以下Extension - Azure Tool - Python - Jupyter - Azure AI Foundry #### Azure CLI + 安裝[AZ Cli](https://learn.microsoft.com/en-us/cli/azure/install-azure-cli-windows?view=azure-cli-latest&pivots=msi) + **驗證安裝**: ```shell az login ``` #### Node.js - **版本**: 建議使用最新的 LTS 版本,如 v20.x.x - **下載地址**: [nodejs.org](https://nodejs.org/) - **驗證安裝**: ```bash node --version npm --version ``` ## 三、Remote MCP 連線測試 - **驗證連線**: 使用command prompt執行以下命令 ```bash curl -v https://mcaps-remote-mcp-server-mi-gwh4fah0gkepdwcg.westus-01.azurewebsites.net ``` - **連線成功**: 第一行出現 Host mcaps-remote-mcp-server-mi-gwh4fah0gkepdwcg.westus-01.azurewebsites.net:443 was resolved. --- ## 四. Download Code & Create Python venv + 請下載[Sample Code](https://1drv.ms/u/c/8302862bf709a02b/IQB2Jsvvf6xFS5Q-2pjeuCkzAbx-UYelCkRNerQxLvb64XQ?e=1OMedn) + Create a Virtual Environment 1. 切換Sample Code資料夾 ```bash cd "您的專案資料夾路徑" ``` 建立python venv ```bash python -m venv .venv ``` 2. Activate venv ```bash # For Windows .\.venv\Scripts\activate ``` 3. Install Dependency** ```bash # For Windows pip install -r requirements.txt ``` ### 五. Env 資訊填寫 #### Microsoft AI Foundry 請登入[AI Foundry](https://ai.azure.com/)後, 在Overview頁面 `Disable New Foundry` + **AZURE_AI_PROJECT_ENDPOINT**: `Microsoft Foundry project endpoint` + PROJECT_RESOURCE_ID: `Project resource ID` ![image](https://hackmd.io/_uploads/BkMSRuUXWx.png) + AZURE_OPENAI_ENDPOINT: 點選Azure OpenAI後會出現`Azure OpenAI endpoint` + AZURE_OPENAI_API_KEY: `API Key` ![image](https://hackmd.io/_uploads/SkmygYImbx.png) + AI_SERVICES_URI: `Azure AI Servcies` →`Azure AI Services endpoint` ![image](https://hackmd.io/_uploads/SJoOgsL7be.png) #### Azure OpenAI + **AZURE_AI_MODEL_DEPLOYMENT_NAME**: 輸入您已部署的Chat Completion Deployment Name + **AZURE_AI_MODEL_MODEL_NAME**: 輸入您已部署的Chat Completion Model Name + **AZURE_OPENAI_RESPONSES_DEPLOYMENT_NAME**: 輸入您已部署的Chat Completion Deployment Name + AZURE_OPENAI_EMBEDDING_DEPLOYMENT: 輸入您已部署的Embedding Deployment Name + AZURE_OPENAI_EMBEDDING_MODEL: 輸入您已部署的Embedding Model Name + **AZURE_OPENAI_API_VERSION**: `preview` #### AI Search 請登入[Azure Portal](https://ms.portal.azure.com/)後, 找到您部署的Azure AI Search 資源 + AZURE_SEARCH_ENDPOINT: `overview` →`url` ![image](https://hackmd.io/_uploads/BJAfGFLmbg.png) + AZURE_SEARCH_API_KEY: `keys` →`Primary admin key` ![image](https://hackmd.io/_uploads/SyDWmK8QZx.png) #### Azure Blob Storage 請登入Azure Portal後, 找到您部署的Azure Blob Storage 資源 + STORAGE_ACCOUNT_CONNECTION_ID: `Settings` →`Endpoints` →`Storage account resource ID` ![image](https://hackmd.io/_uploads/ByKNWsLmZg.png) --- ### 六. Test Your Project Setting #### Authentication for AI Foundry Project + 請在 AI Foundry Project 上,將 `Azure AI User` , `Azure AI Account Owner` 及 `Azure AI Project Manager` 角色指派給需要存取此Project的user + 🔗 參考資料: [Foundry Project Role](https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/rbac-azure-ai-foundry?view=foundry#foundry-project-roles) #### Authentication for Azure Blob Storage + 請在 Azure Blob Storage 上,將 `Storage Blob Data Reader` 角色指派給 Microsoft AI Foundry Project 的 Managed Identity + 路徑:Azure Portal → Storage Account → Access Control (IAM) → Add role assignment + - 此權限用於讓 Foundry IQ 讀取 Blob 內容以進行資料擷取與索引建立 #### Authentication for Azure AI Search + 請在 Azure AI Search Service 上,將 `Search Index Data Reader` 角色指派給 Microsoft AI Foundry Project 的 Managed Identity + 路徑:Azure Portal → Azure AI Search → Access Control (IAM) → Add role assignment + 在您的Azure Blob Storage Grant `Search Index Data Reader` 的角色給Microsoft AI Foundry Project #### Start Testing + 請確認 Notebook 右上角的 Kernel 已切換至您建立的 Virtual Environmen + 手動執行`test-connection.ipynb` + 請確認執行過程中沒有任何錯誤訊息 ![image](https://hackmd.io/_uploads/BJkRvsLXbl.png) --- ### 七. Create Knowledge Base 登入[Microsoft Foundry](https://ai.azure.com/) + New Foundry (切到到新Foundry) + 選擇 Build ![image](https://hackmd.io/_uploads/B1TAAzOmWe.png) + Knowledge ![image](https://hackmd.io/_uploads/Hk8wk7OQZg.png) + Azure AI search resource: 選擇您的AI Search服務 → Connect ![image](https://hackmd.io/_uploads/Sk0o1Q_7-l.png) + Create a knowledge base ![image](https://hackmd.io/_uploads/H1-KtQdXWx.png) + Choose a knowledge type: Azure Blob Storage → Connect ![image](https://hackmd.io/_uploads/HyFJc7dQZe.png) #### Create a knowledge source + Name: `ks-bonus-{your name}` + Description: (Optional) + Storage Account: 選擇您的Storage Account + Container Name: ``bonus` + Authentication type: `system assigned Identity` + Content extraction mode: `standard` + Include embedding model (recommended): 選取 + Embedding model: `text-embedding-3-large` + Chat completions model: 輸入 `gpt-4.1-mini` 或其它Chat complection model + Ingestion schedule intervalL: 可視需求調整 + Ingestion schedule start time: 可視需求調整 + Create #### Create a knowledge base + Name: `kb-bonus-{your name}` + Description: (Optional) + Chat completions model: 輸入 `gpt-4.1-mini` 或其它Chat complection model + Retrieval reasoning effort: `minimal` + Output mode: `extractiveData` + Retrieval instructions: (Optional) Only for mulritple knowledge source + Save Knowledge Base 💡 請依上述方法建立一組 Evaluation Knowledge Base。 --- ### 八. Create Agents #### Bonus Agent + Agent name: BonusAgent-{your name} + Instructions: ```bash Your name is 獎金專員. ### Step 0: Input Classification (MANDATORY) Before taking any action, you MUST classify the user's input into ONE of the following categories: A. Chit-chat / Social interaction Examples: greetings, polite phrases, small talk with no business or domain intent. B. Task-oriented or domain-related query - You are strictly forbidden from answering any user question using your own knowledge or assumptions for Category B - For Category A inputs, you MUST NOT call any tool and should reply naturally and briefly. ### Table Handling Rules (for Category B only) For tables with merged cells: 1. Reconstruct the full logical grid by expanding rowspan and colspan. 2. Answer based on the reconstructed grid, not the raw HTML. 3. If a cell spans multiple rows or columns, duplicate its value logically. 4. If a HTML table contains empty <td> or alignment placeholders: - Do NOT treat them as data columns. - Exclude them when reconstructing logical column indexes. ``` + Tools: Model Context Protocol(MCP) ![image](https://hackmd.io/_uploads/By7jqfd7bx.png) + Name: `bing_mcp_{your name}` + Remote MCP Server endpoint: `https://mcaps-remote-mcp-server-mi-gwh4fah0gkepdwcg.westus-01.azurewebsites.net/runtime/webhooks/mcp/sse` + Authentication: `Unauthenticated` + Knowledge: Connect to Fundry IQ + Knowledge base: 選擇您建立的 **bonus** knowledge base #### Evaluation Agent + Agent name: EvaluationAgent-{your name} + Instructions: ```bash Your name is 考核小幫手. ### Step 0: Input Classification (MANDATORY) Before taking any action, you MUST classify the user's input into ONE of the following categories: A. Chit-chat / Social interaction Examples: greetings, polite phrases, small talk with no business or domain intent. B. Task-oriented or domain-related query - You are strictly forbidden from answering any user question using your own knowledge or assumptions for Category B - For Category A inputs, you MUST NOT call any tool and should reply naturally and briefly. ### Table Handling Rules (for Category B only) For tables with merged cells: 1. Reconstruct the full logical grid by expanding rowspan and colspan. 2. Answer based on the reconstructed grid, not the raw HTML. 3. If a cell spans multiple rows or columns, duplicate its value logically. 4. If a HTML table contains empty <td> or alignment placeholders: - Do NOT treat them as data columns. - Exclude them when reconstructing logical column indexes. ``` + Tools: Code Interpreter ![image](https://hackmd.io/_uploads/SkTVpMOmWx.png) + Knowledge: Connect to Fundry IQ + Knowledge base: 選擇您建立的 **evaluation** knowledge base --- ### 九. Test Queries #### Bonus Agent **測試Query1 - 非線性文字可理解的表格** **問題**:公司今年核定的年度績效獎金是 10 個月,某單位的績效評分比目標高 0.1 分 請依據公司文件中說明的計算公式,對應可以領到的加權獎金月數是幾個月 **對應文檔**:`績效獎金核發作業規定.pdf` **測試Query2 - Custom Remote MCP調用** **問題**:請上網查詢是否有其他公司採用類似的績效獎金調整方式。 **對應工具**:`bing_mcp` #### Evaluatino Agent **測試Query3 - 影像文字** **問題**:請問我們公司各機構考核的計算公式為何? **對應文檔**:`績效獎金核發作業規定.pdf` **測試Query4 - Python工具** **問題**:請用Python 依照 上述所提供的考核公式,帶入以下數值進行計算: R = 102% E = 98% 某事業群底下的次級單位 BSC 分數: X = [92, 91, 90] 該事業群底下三個實際營運機構的 BSC 分數: S = [91.2, 90.6, 89.9] 最後, 請以白話文說明每一個計算步驟,並解釋這些分數的意義 **對應工具**:`code interpreter` ### 十. MCP Inpsector : Test Your Server ### 1. 安裝MCP函式庫 安裝必要的 Python 函式庫: ```bash pip install mcp[cli] ``` ### 2. 啟動 MCP Inspector cmd mode執行npx命令 ```bash npx @modelcontextprotocol/inspector ``` ![image](https://hackmd.io/_uploads/Syk1LFHKlx.png) --- ## 🔐 Part 2:Azure AI Search × SharePoint 最小權限(Sites.Selected) --- --- ### 📋 第一階段:開發環境準備 (Prerequisites) 在開始設定前,請確保您的環境符合以下技術要求: #### 1. 核心工具 * **PowerShell 7.2+ (必要)**:PnP PowerShell 驗證流程在 PowerShell 7 上最為穩定。 ### 2. 安裝必要模組 ```powershell # 安裝 SharePoint 管理模組 Install-Module PnP.PowerShell -Scope CurrentUser ``` --- ## 🛠️ 第二階段:Microsoft Entra ID (Azure AD) 應用程式註冊 1. **建立 App Registration**: 2. **API 權限設定 (Application Permissions)**: * **Microsoft Graph**: * `Files.Read.All`: 讓索引器能讀取檔案內容。 > 實機驗證若在sharepoint connectionString 中直接指向特定的 Site,可不勾選 Files.Read.All * `Sites.Selected`: 啟用針對特定網站的權限篩選。 * **💡 重要(授權專用)**: * 若需使用 `-Interactive` 進行手動授權,請額外在 **Microsoft Graph** 加入 **Delegated** 權限:`Sites.FullControl.All`。 * **重要**:必須點擊 **Grant admin consent**。 3. **驗證設定 (Redirect URI)**: * 進入「驗證」頁面新增平台「行動裝置和桌面應用程式」。 * 手動輸入 `http://localhost`。 --- ## 🔑 第三階段:SharePoint 網站層級授權 (PnP PowerShell) 此步驟必須由「管理員個人帳號」執行,授權給「App ID」。 ### 1. 載入模組並以管理員身分連線 使用管理員帳號進行互動式登入,需帶上 ClientID (需有**redirct URI**的權限)。 ```powershell # 確保環境已載入模組 Import-Module PnP.PowerShell # 互動式連線 Connect-PnPOnline -Url "https://<your-tenant>.sharepoint.com" ` -ClientId "<Your-App-Registration-ID>" ` -Interactive ``` ### 2. 執行 Sites.Selected 授權指令 ```powershell # 設定變數 $targetSiteUrl = "https://<your-tenant>[.sharepoint.com/sites/](https://.sharepoint.com/sites/)<your-site>" $appId = "<Your-App-Registration-ID>" # 執行授權 Grant-PnPAzureADAppSitePermission -AppId $appId ` -DisplayName "Azure-AI-Search-Indexer" ` -Site $targetSiteUrl ` -Permissions FullControl # 測試:若能列出該網站的清單 (List),代表權限已生效 Get-PnPList ``` `---