---
title: RAI for Wistron
tags: [RAI, wistron]

---

# Azure AI Foundry Responsible AI


---

## 一、前置需求
+ [Azure AI Foundry環境建立](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/create-projects?tabs=ai-foundry&pivots=fdp-project)
- 目前可用 Region: East US2, Sweden Central, Switzerland West
- 請使用Foundry Project (非Hub base Project)
+ [Private Link for AI Foudndry](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/configure-private-link?tabs=azure-portal&pivots=fdp-project)
- PoC階段不會使用到Agent Services
+ Python Virtual Environemnt
- 請安裝相關packages

   pip install -r requirements.txt

---
## 測試1 - Prompt Injection (REST API)
+ 參考代碼: 1 PromptShiled.ipynb
+ 在AI Foundry Project 取得 API Enpoint 及 AI Key
+ 準備惡意Prompt進行測試
+ Output
---
## 測試2. Improper Output & RAG Reference
+ 參考代碼: 2 ImproperOutput & RAG Reference .ipynb
+ 在AI Foundry Project 取得 Project Enpoint
+ Azure OpenAI Endpoint 及 Azure OpenAI Key
+ Evaluator測項請依需求自行調整
**General purpose**:`CoherenceEvaluator`, `FluencyEvaluator`
**Textual similarity**: `SimilarityEvaluator`
**RAG**: `RetrievalEvaluator`, `GroundednessEvaluator`, `RelevanceEvaluator`
**Risk and safety**: `IndirectAttackEvaluator`, `UngroundedAttributesEvaluator`, `ContentSafetyEvaluator`

+ 測試資料集(csv或JSONL): 請置換為您的資料集名稱
+ 測試資料集需包含下以下四個欄位
+ query: 使用者的問題
+ response: LLM的回覆
+ ground_truth: 標準答案
+ context: 對應的文本上下文

+ Azure Foundry Portal Dashboard
+ 點擊 Evaluation
+ 選取 Automated evaluations並選取最近一筆
+ 相關指標: 可切換 `AI quality (AI Assisted)` 及 `Risk and safety (preview)`

---
## 測試3. Red Teaming
+ 參考代碼: 3 AI Red Teaming locally.ipynb
+ 在AI Foundry Project 取得 Project Enpoint
+ 功擊目標 LLM: 請將 ask_LLM 函式替換為符合您需求的目標模型。本範例程式碼，展示如何攻擊 Azure OpenAI 模型。請確保您已提供 Azure OpenAI 的 Endpoint、API Key 以及 Model Name
+ [Risk Categories](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent#supported-risk-categories): 依測試需求選擇對應的風險類
+ num_objectives: 依測試需求調整各別對應的`功擊次數`
+ [attack_straregies](https://learn.microsoft.com/en-us/azure/ai-foundry/how-to/develop/run-scans-ai-red-teaming-agent#specific-attack-strategies): 依測試需求調整各別對應的`功擊策略`

+ Azure Foundry Portal Dashboard
+ 點擊 Evaluation
+ 選取 AI red teaming並選取最近一筆
+ 相關指標