Log Analytics 工作區

# Log Analytics 工作區 ###### tags: `AZURE 功能` * 如果要將 Log 放進 Log Analytics 查詢工作區中，需要在各雲資源中診斷設定中設定，把需要的 Log 傳到你要的工作區 ### 查詢類別 * AzureDiagnostics : 儲存使用了 Azure 診斷模式的 Azure 服務資源記錄 * AzureMetrics : Azure 服務發出的計量資料，可測量其健康情況和效能 * Usage : 工作區中每個資料表的每小時使用量資料 (查詢 worksapce 的使用情況，基本上不會用到) ### Log 類別 * FrontDoorAccessLog : 存取記錄檔 * FrontDoorHealthProbeLog : 健全狀態探查記錄，會提供每次失敗探查的記錄 * FrontDoorWebApplicationFirewallLog : 有被 WAF 擋下來才會有紀錄，無流量的 byte 值，無法獲取阻擋的流量 LOG * AllMetrics : 資源的平台計量紀錄 #### 查詢所需 Log ``` AzureDiagnostics | where Category !="FrontDoorHealthProbeLog" //查詢健康檢測以外的 Log ``` #### 某時間內的紀錄 ``` AzureDiagnostics | where TimeGenerated > ago(30d) //某時間之內 ``` #### 過去某時間內以某區間為單位的連線次數 ``` AzureDiagnostics | where TimeGenerated > ago(30d) | where Category == "FrontDoorAccessLog" | summarize count() by bin(TimeGenerated, 1h) // 時間區段單位 ``` #### 查詢 CDN 回應的流量 ``` AzureDiagnostics | where Category == "FrontDoorAccessLog" | summarize Response_Bytes = sum(toint(responseBytes_s)) by requestUri_s // 每筆資料的流量加總 | order by Response_Bytes ``` #### 查詢 CDN 請求的流量 ``` AzureDiagnostics | where Category == "FrontDoorAccessLog" | summarize Request_Bytes = sum(toint(requestBytes_s)) by requestUri_s // 每筆資料的流量加總 | order by Request_Bytes ``` #### 查詢個別 URL 的請求流量 ``` AzureDiagnostics | where isnotempty(requestUri_s) | where Category != "FrontDoorHealthProbeLog" | summarize Request_Bytes = sum(toint(requestBytes_s)) by requestUri_s | order by Request_Bytes ``` #### 查詢總共請求的流量 ``` AzureDiagnostics | where Category != "FrontDoorHealthProbeLog" | summarize RequestBytes = sum(toint(requestBytes_s)) by requestUri_s contains "<欲查詢域名>" | where Column1 == true ``` #### 查詢以一分鐘為單位請求的流量 ``` AzureDiagnostics | where Category != "FrontDoorHealthProbeLog" | summarize RequestBytes = sum(toint(requestBytes_s)) by requestUri_s contains "sprint-cdn.com" , bin(TimeGenerated, 60s) | where Column1 == true ``` #### 查詢一分鐘內平均請求的流量 ``` AzureDiagnostics | where TimeGenerated > ago(60s) | where Category != "FrontDoorHealthProbeLog" | summarize RequestBytes = sum(toint(requestBytes_s))/60 by requestUri_s contains "sprint-cdn.com" | where Column1 == true ``` #### 查詢某時間內某單位平均請求的流量 ``` AzureDiagnostics | where TimeGenerated > ago(30d) | where Category != "FrontDoorHealthProbeLog" | summarize Request_Bytes = sum(toint(requestBytes_s)) by requestUri_s contains "sprint-cdn.com" , bin(TimeGenerated, 1h) | where Column1 == true ``` #### IP 連線數熱門前十排行 ``` AzureDiagnostics | where Category != "FrontDoorHealthProbeLog" | where isnotempty(clientIP_s) //不包含空值 | summarize IP_Connection_Times = count() by clientIP_s | top 10 by IP_Connection_Times ``` #### Domain 連線數熱門前十排行 ``` AzureDiagnostics | where Category != "FrontDoorHealthProbeLog" | where isnotempty(domain_s) //不包含空值 | summarize Domain_Connection_Times = count() by domain_s | top 10 by Domain_Connection_Times ``` #### URL 連線數熱門前十排行 ``` AzureDiagnostics | where isnotempty(requestUri_s) //不包含空值 | where Category != "FrontDoorHealthProbeLog" | summarize URL_Connection_Times = count() by requestUri_s | top 10 by URL_Connection_Times ``` #### 查詢 Front Door WAF 封鎖的 log ``` AzureDiagnostics | where Category == "FrontDoorWebApplicationFirewallLog" | where action_s == "Block" //動作為被封鎖 ``` #### 查看規則符合 WAF 規則的連線數的用戶 IP ``` AzureDiagnostics | where Category == "FrontDoorWebApplicationFirewallLog" | summarize Block_IP_Count = count() by clientIP_s | top 10 by Block_IP_Count ``` #### 查看被 WAF 拒絕的用戶 IP 的統計 ``` AzureDiagnostics | where Category == "FrontDoorWebApplicationFirewallLog" | where action_s == "Block" | summarize Request_Count = count() by ClientIP = clientIP_s, RuleName = ruleName_s | top 10 by Request_Count ``` #### 查詢最多用戶訪問的國家地區 ``` AzureDiagnostics | where TimeGenerated > ago(30d) | where isnotempty( clientCountry_s) | where Category == "FrontDoorAccessLog" | summarize Client_Connection_Country = count() by clientCountry_s | top 10 by Client_Connection_Country ``` #### 每個時段的請求次數 ``` AzureDiagnostics | where TimeGenerated > ago(30d) | where Category == "FrontDoorAccessLog" | summarize RequestCount = count() by bin(TimeGenerated, 5m) ``` #### 每個時段用戶 IP 被 WAF 拒絕的次數 ``` AzureDiagnostics | where TimeGenerated > ago(30d) | where Category == "FrontDoorWebApplicationFirewallLog" and action_s == "Block" | summarize count() by clientIp_s, bin(TimeGenerated, 1m) ``` #### 查詢 URL 的次數 ``` AzureDiagnostics | where isnotempty(requestUri_s) | where Category == "FrontDoorAccessLog" | summarize count() by requestUri_s ``` #### 查詢各 IP 的 URL 的次數 ``` AzureDiagnostics | where isnotempty(requestUri_s) and isnotempty(clientIP_s) | where Category == "FrontDoorAccessLog" | summarize count() by requestUri_s,clientIP_s ``` #### 查詢 LB 流量每分鐘 (記得要切存流量的 LOG 工作區) ``` AzureMetrics | where TimeGenerated > ago(30d) | summarize Flow = sum(toint(Total)) by Resource , bin(TimeGenerated, 1h) | render timechart ``` ### 時間區段 * TimeGenerated : 生成時間 * TimeReceived : 收到時間 * IngestionTime : 攝入時間 #### 區段時間範圍內的資料 ``` AzureDiagnostics | where TimeGenerated between(startofweek(ago(30days))..endofweek(ago(1days))) | where Category !="FrontDoorHealthProbeLog" | summarize count() by bin(TimeGenerated, 1h) | order by TimeGenerated asc ``` ## Azure cli #### Format ``` # az monitor log-analytics query \ --workspace <WORKSPACE-ID> \ --analytics-query "查詢函式" ``` #### Example ``` # az monitor log-analytics query \ --workspace e45f6e53-2add-4dbd-91c7-bdd42abf0575 \ --analytics-query "AzureDiagnostics | where Category == 'FrontDoorAccessLog' | summarize Request_Bytes = sum(toint(requestBytes_s)) by requestUri_s | order by Request_Bytes" ``` #### Final ``` # az monitor log-analytics query \ --workspace e45f6e53-2add-4dbd-91c7-bdd42abf0575 \ --analytics-query "AzureDiagnostics | where Category == 'FrontDoorAccessLog' | summarize Request_Bytes = sum(toint(requestBytes_s)) by requestUri_s | order by Request_Bytes" | jq -r '.[]' { "Request_Bytes": "428208", "TableName": "PrimaryResult", "requestUri_s": "https://ddos.sprint-cdn.com:443/" } { "Request_Bytes": "9937", "TableName": "PrimaryResult", "requestUri_s": "http://ddos.sprint-cdn.com:80/" } { "Request_Bytes": "6240", "TableName": "PrimaryResult", "requestUri_s": "https://ddos.sprint-cdn.com:443/favicon.ico" } { "Request_Bytes": "1466", "TableName": "PrimaryResult", "requestUri_s": "https://walle-ss-h6cbevdjg3bmb9h7.z01.azurefd.net:443/" } { "Request_Bytes": "1348", "TableName": "PrimaryResult", "requestUri_s": "https://walle-ss-h6cbevdjg3bmb9h7.z01.azurefd.net:443/favicon.ico" } { "Request_Bytes": "496", "TableName": "PrimaryResult", "requestUri_s": "http://walle-ss-h6cbevdjg3bmb9h7.z01.azurefd.net:80/" } { "Request_Bytes": "288", "TableName": "PrimaryResult", "requestUri_s": "https://ddos.sprint-cdn.com:443/favicon.png" } { "Request_Bytes": "0", "TableName": "PrimaryResult", "requestUri_s": "" } ``` #### 顯示工作區資訊 ``` az monitor log-analytics workspace show \ --resource-group Protector01 \ --name Protector01-FD-Log \ --subscription 4cdff114-7f4d-46d1-9a6c-ad0c43eb0a5a ``` ``` { "createdDate": "2023-02-20T02:22:52.0193249Z", "customerId": "f306d409-37be-4053-8bc6-3bceefca91e0", "features": { "enableLogAccessUsingOnlyResourcePermissions": true }, "id": "/subscriptions/4cdff114-7f4d-46d1-9a6c-ad0c43eb0a5a/resourceGroups/Protector01/providers/Microsoft.OperationalInsights/workspaces/Protector01-FD-Log", "location": "eastasia", "modifiedDate": "2023-02-20T02:22:52.0193249Z", "name": "Protector01-FD-Log", "provisioningState": "Succeeded", "publicNetworkAccessForIngestion": "Enabled", "publicNetworkAccessForQuery": "Enabled", "resourceGroup": "Protector01", "retentionInDays": 30, "sku": { "lastSkuUpdate": "2023-02-20T02:22:52.0193249Z", "name": "pergb2018" }, "tags": {}, "type": "Microsoft.OperationalInsights/workspaces", "workspaceCapping": { "dailyQuotaGb": -1.0, "dataIngestionStatus": "RespectQuota", "quotaNextResetTime": "2023-02-20T06:00:00Z" } } ``` #### 顯示工作區資訊 (查詢工作區 ID) ``` az monitor log-analytics workspace show \ --resource-group Protector01 \ --name Protector01-FD-Log \ --subscription 4cdff114-7f4d-46d1-9a6c-ad0c43eb0a5a | jq -r '.customerId' ``` ``` f306d409-37be-4053-8bc6-3bceefca91e0 ```