Install Istio with ambient mode (使用指令安裝)

# Install Istio with ambient mode (使用指令安裝) ## 安裝 istio 前需要先安裝 prometheus 跟 grafana ### 安裝 helm3 ``` $ curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash $ helm version version.BuildInfo{Version:"v3.17.1", GitCommit:"980d8ac1939e39138101364400756af2bdee1da5", GitTreeState:"clean", GoVersion:"go1.23.5"} ``` 安裝 prometheus 跟 grafana ``` $ kubectl create ns monitoring-system $ helm repo add prometheus-community https://prometheus-community.github.io/helm-charts $ helm repo update ``` * 配置 prometheus helm value 檔，可以把 alertmanager 關閉。 * 設定 kubelet 不要 drop 任何的 metric ``` $ nano prometheus.yaml alertmanager: enabled: false kubelet: enabled: true serviceMonitor: cAdvisorMetricRelabelings: [] $ helm install monitoring prometheus-community/kube-prometheus-stack --namespace monitoring-system -f prometheus.yaml ``` 確認部屬完成 ``` $ kubectl -n monitoring-system get pod NAME READY STATUS RESTARTS AGE monitoring-grafana-789c7768b8-zs958 3/3 Running 0 65s monitoring-kube-prometheus-operator-78c5549fff-b8fhn 1/1 Running 0 65s monitoring-kube-state-metrics-557898b467-mv8z2 1/1 Running 0 65s monitoring-prometheus-node-exporter-6r2bh 1/1 Running 0 65s monitoring-prometheus-node-exporter-vk7n4 1/1 Running 0 65s monitoring-prometheus-node-exporter-zmd96 1/1 Running 0 65s prometheus-monitoring-kube-prometheus-prometheus-0 1/2 Running 0 49s ``` ``` $ helm -n monitoring-system list NAME NAMESPACE REVISION UPDATED STATUS CHART APP VERSION monitoring monitoring-system 1 2025-07-03 10:56:47.390660705 +0800 CST deployed kube-prometheus-stack-75.2.1 v0.83.0 ``` ## 下載 Istio CLI ``` $ curl -L https://istio.io/downloadIstio | sh - $ sudo cp istio-1.26.1/bin/istioctl /usr/local/bin $ istioctl version Istio is not present in the cluster: no running Istio pods in namespace "istio-system" client version: 1.26.1 ``` ## Install the Kubernetes Gateway API CRDs 請注意，大多數 Kubernetes 叢集預設並未安裝 Kubernetes Gateway API 的 CRD，因此在使用 Gateway API 之前，請先確保這些 CRD 已安裝： ``` $ kubectl get crd gateways.gateway.networking.k8s.io &> /dev/null || \ kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml ``` * 檢查 CRD 是否安裝成功 ``` $ kubectl get -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml NAME CREATED AT gatewayclasses.gateway.networking.k8s.io 2025-06-19T02:18:11Z gateways.gateway.networking.k8s.io 2025-06-19T02:18:11Z grpcroutes.gateway.networking.k8s.io 2025-06-19T02:18:11Z httproutes.gateway.networking.k8s.io 2025-06-19T02:18:11Z referencegrants.gateway.networking.k8s.io 2025-06-19T02:18:12Z ``` ## Install Istio on to your cluster `istioctl` 支援多種設定範本，這些範本包含不同的預設選項，並可依照您的生產需求進行自訂。`ambient` 模式的支援已包含在 `ambient` 範本中。使用以下指令安裝 Istio： ``` # 設定啟用 trace 功能，指標提供給 jaeger $ cat <<EOF > ./tracing.yaml apiVersion: install.istio.io/v1alpha1 kind: IstioOperator spec: profile: ambient meshConfig: enableTracing: true defaultConfig: tracing: tracing: {} # disable legacy MeshConfig tracing options extensionProviders: - name: jaeger opentelemetry: service: jaeger-collector.istio-system.svc.cluster.local port: 4317 EOF $ istioctl install -f ./tracing.yaml --skip-confirmation ``` * 檢查 Istio pods 狀態是否正常 ``` $ kubectl -n istio-system get pods NAME READY STATUS RESTARTS AGE istio-cni-node-bzlwz 1/1 Running 0 6m31s istio-cni-node-nz6z7 1/1 Running 0 6m31s istio-cni-node-rhhls 1/1 Running 0 6m31s istio-cni-node-tf26z 1/1 Running 0 6m31s istio-cni-node-tjg9h 1/1 Running 0 6m31s istio-cni-node-vn26k 1/1 Running 0 6m31s istiod-799b8c5498-ntgmg 1/1 Running 0 7m1s ztunnel-4zfgj 1/1 Running 0 4m29s ztunnel-9c8pp 1/1 Running 0 4m29s ztunnel-9mr7v 1/1 Running 0 4m29s ztunnel-mxbgq 1/1 Running 0 4m29s ztunnel-pjlps 1/1 Running 0 2m5s ztunnel-z2pr2 1/1 Running 0 4m29s ``` * `istio-cni-node` : 負責在每個 Node 上自動設置網路規則（例如 iptables/ebpf），將流量導向 Ambient Mesh 處理邏輯。 * `istiod` : Istio 控制平面主服務，負責管理整個 Istio Mesh。 * `ztunnel` : Ambient 模式的核心組件，用來取代 Envoy Sidecar，負責處理 Node 上所有 Pod 的入出流量。 * 注意: `ztunnel` 會需要使用 ipv6 ## 安裝 kiali ``` $ kubectl apply -f https://raw.githubusercontent.com/cooloo9871/Kiali/refs/heads/main/kiali.yaml ``` ``` $ kubectl -n istio-system get pod -l app.kubernetes.io/instance=kiali NAME READY STATUS RESTARTS AGE kiali-7cbdf5689-bccnh 1/1 Running 0 2m51s ``` 透過 NodePort 連到 kiali 服務 ``` $ kubectl -n istio-system get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-ingressgateway NodePort 10.98.0.146 <none> 80:30962/TCP,443:31894/TCP 69s istiod ClusterIP 10.98.0.182 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 18m kiali NodePort 10.98.0.22 <none> 20001/TCP,9090/TCP 16m ``` 也可以設定 kiali 服務對外，指定本機網卡 ip ``` $ kubectl port-forward --address 10.10.7.30 svc/kiali 20001:20001 -n istio-system ``` ## 安裝 ingressgateway * `istio-ingressgateway` `service` type 是 `NodePort` ，可以根據環境需求更改 ``` $ nano ingress.yaml apiVersion: v1 kind: Service metadata: name: istio-ingressgateway namespace: istio-system labels: istio: ingressgateway spec: type: NodePort selector: istio: ingressgateway ports: - port: 80 name: http - port: 443 name: https --- apiVersion: apps/v1 kind: Deployment metadata: name: istio-ingressgateway namespace: istio-system spec: selector: matchLabels: istio: ingressgateway template: metadata: annotations: # Select the gateway injection template (rather than the default sidecar template) inject.istio.io/templates: gateway labels: # Set a unique label for the gateway. This is required to ensure Gateways can select this workload istio: ingressgateway # Enable gateway injection. If connecting to a revisioned control plane, replace with "istio.io/rev: revision-name" sidecar.istio.io/inject: "true" spec: # Allow binding to all ports (such as 80 and 443) securityContext: sysctls: - name: net.ipv4.ip_unprivileged_port_start value: "0" containers: - name: istio-proxy image: auto # The image will automatically update each time the pod starts. # Drop all privileges, allowing to run as non-root securityContext: capabilities: drop: - ALL runAsUser: 1337 runAsGroup: 1337 --- # Set up roles to allow reading credentials for TLS apiVersion: rbac.authorization.k8s.io/v1 kind: Role metadata: name: istio-ingressgateway-sds namespace: istio-system rules: - apiGroups: [""] resources: ["secrets"] verbs: ["get", "watch", "list"] --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: istio-ingressgateway-sds namespace: istio-system roleRef: apiGroup: rbac.authorization.k8s.io kind: Role name: istio-ingressgateway-sds subjects: - kind: ServiceAccount name: default ``` ``` $ kubectl apply -f ingress.yaml $ kubectl -n istio-system get pod -l istio=ingressgateway NAME READY STATUS RESTARTS AGE istio-ingressgateway-85444c49f5-4zfq8 1/1 Running 0 76s ``` ## 建立 servicemonitor * 部屬 servicemonitor 讓 Prometheus 可以收集到 istio 的相關指標 * 在 Kubernetes 上抓取指標是透過 serviceMonitor 、podMonitor 設定⽬標服務所提供的指標並存入後端資料庫，兩者差別 serviceMonitor 是抓取 Service 對應 Endpoints 上的監控數據，⽽ podMonitor 是 Pod 上對應的數據，⼀般來說是使⽤ serviceMonitor，除非只有抓取單⼀ Pod 資訊需求。 ``` $ nano servicemonitor.yaml apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: istio-ingressgateway namespace: istio-system labels: release: monitoring spec: selector: matchLabels: istio: ingressgateway namespaceSelector: matchNames: - istio-system endpoints: - targetPort: http-envoy-prom path: /stats/prometheus --- apiVersion: monitoring.coreos.com/v1 kind: ServiceMonitor metadata: name: istiod namespace: istio-system labels: release: monitoring spec: selector: matchLabels: istio: pilot namespaceSelector: matchNames: - istio-system endpoints: - port: http-monitoring interval: 15s ``` 檢查部屬 ``` $ kubectl apply -f servicemonitor.yaml $ kubectl -n istio-system get servicemonitor NAME AGE istio-ingressgateway 5m55s istiod 5m55s ``` * 需注意 `release: monitoring` label ，這邊要填寫在 Prometheus ，設定要收集的對象是誰，如果在 `ruleSelector` 檢查看到對應的 label，那麼在建立 servicemonitor 時也須貼上相同 label，才會被納入使用。 ``` $ kubectl -n monitoring-system get prometheuses monitoring-kube-prometheus-prometheus -o yaml | grep -A 2 serviceMonitorSelector serviceMonitorSelector: matchLabels: release: monitoring ``` ## 安裝 Jaeger Jaeger 可以對請求流量進行跟蹤，是微服務中複雜請求的追蹤方案 ``` $ nano jaeger.yaml apiVersion: apps/v1 kind: Deployment metadata: name: jaeger namespace: istio-system labels: app: jaeger spec: selector: matchLabels: app: jaeger template: metadata: labels: app: jaeger sidecar.istio.io/inject: "false" annotations: prometheus.io/scrape: "true" prometheus.io/port: "14269" spec: containers: - name: jaeger image: "docker.io/jaegertracing/all-in-one:1.65.0" env: - name: BADGER_EPHEMERAL value: "false" - name: SPAN_STORAGE_TYPE value: "badger" - name: BADGER_DIRECTORY_VALUE value: "/badger/data" - name: BADGER_DIRECTORY_KEY value: "/badger/key" - name: COLLECTOR_ZIPKIN_HOST_PORT value: ":9411" - name: MEMORY_MAX_TRACES value: "50000" - name: QUERY_BASE_PATH value: /jaeger livenessProbe: httpGet: path: / port: 14269 readinessProbe: httpGet: path: / port: 14269 volumeMounts: - name: data mountPath: /badger resources: requests: cpu: 10m volumes: - name: data emptyDir: {} --- apiVersion: v1 kind: Service metadata: name: tracing namespace: istio-system labels: app: jaeger spec: type: NodePort ports: - name: http-query port: 80 protocol: TCP targetPort: 16686 # Note: Change port name if you add '--query.grpc.tls.enabled=true' - name: grpc-query port: 16685 protocol: TCP targetPort: 16685 selector: app: jaeger --- # Jaeger implements the Zipkin API. To support swapping out the tracing backend, we use a Service named Zipkin. apiVersion: v1 kind: Service metadata: labels: name: zipkin name: zipkin namespace: istio-system spec: ports: - port: 9411 targetPort: 9411 name: http-query selector: app: jaeger --- apiVersion: v1 kind: Service metadata: name: jaeger-collector namespace: istio-system labels: app: jaeger spec: type: ClusterIP ports: - name: jaeger-collector-http port: 14268 targetPort: 14268 protocol: TCP - name: jaeger-collector-grpc port: 14250 targetPort: 14250 protocol: TCP - port: 9411 targetPort: 9411 name: http-zipkin - port: 4317 name: grpc-otel - port: 4318 name: http-otel selector: app: jaeger $ kubectl apply -f jaeger.yaml ``` istio 要啟用 trace 功能 ``` $ kubectl apply -f - <<EOF apiVersion: telemetry.istio.io/v1 kind: Telemetry metadata: name: mesh-default namespace: istio-system spec: tracing: - providers: - name: jaeger EOF ``` ``` $ kubectl -n istio-system get telemetry NAME AGE mesh-default 53s ``` ## 檢查 istio 部屬狀態 ``` $ kubectl -n istio-system get pod NAME READY STATUS RESTARTS AGE istio-cni-node-bzlwz 1/1 Running 0 42m istio-cni-node-nz6z7 1/1 Running 0 42m istio-cni-node-rhhls 1/1 Running 0 42m istio-cni-node-tf26z 1/1 Running 0 42m istio-cni-node-tjg9h 1/1 Running 0 42m istio-cni-node-vn26k 1/1 Running 0 42m istio-ingressgateway-5b6548c6d6-qzcw6 1/1 Running 0 29m istiod-84444f67c-5qh4r 1/1 Running 0 6m22s jaeger-868fbc75d7-q5p9g 1/1 Running 0 98s kiali-6d774d8bb8-l4vrc 1/1 Running 0 32m ztunnel-4zfgj 1/1 Running 0 40m ztunnel-9c8pp 1/1 Running 0 40m ztunnel-9mr7v 1/1 Running 0 40m ztunnel-mxbgq 1/1 Running 0 40m ztunnel-pjlps 1/1 Running 0 38m ztunnel-z2pr2 1/1 Running 0 40m $ kubectl -n istio-system get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-ingressgateway NodePort 10.96.105.108 <none> 80:32105/TCP,443:31605/TCP 34m istiod ClusterIP 10.96.8.191 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 48m jaeger-collector ClusterIP 10.96.187.213 <none> 14268:31068/TCP,14250:31871/TCP,9411:32368/TCP,4317:31106/TCP,4318:30351/TCP 6m34s kiali NodePort 10.96.231.47 <none> 20001/TCP,9090/TCP 37m tracing NodePort 10.96.11.120 <none> 80:31997/TCP,16685:30848/TCP 6m35s zipkin ClusterIP 10.96.65.238 <none> 9411/TCP 6m35s ``` ## 實測金絲雀 workload 流量管理 ## 實作 * 在 test namespace 設定 `istio.io/dataplane-mode=ambient` label ``` $ kubectl create ns test $ kubectl label namespace test istio.io/dataplane-mode=ambient ``` * 部屬 v1 版本 ``` $ echo 'apiVersion: apps/v1 kind: Deployment metadata: name: helloworld-v1 namespace: test spec: replicas: 1 selector: matchLabels: app: helloworld version: v1 template: metadata: labels: app: helloworld version: v1 spec: containers: - name: app image: quay.io/flysangel/image:app.golang' | kubectl apply -f - ``` * 部屬 v2 版本 ``` $ echo 'apiVersion: apps/v1 kind: Deployment metadata: name: helloworld-v2 namespace: test spec: replicas: 1 selector: matchLabels: app: helloworld version: v2 template: metadata: labels: app: helloworld version: v2 spec: containers: - name: app image: quay.io/flysangel/image:app.golang' | kubectl apply -f - ``` * 建立 svc ``` $ echo 'apiVersion: v1 kind: Service metadata: name: helloworld namespace: test spec: selector: app: helloworld ports: - protocol: TCP port: 80 targetPort: 8080' | kubectl apply -f - ``` * 檢查 ``` $ kubectl -n test get all NAME READY STATUS RESTARTS AGE pod/helloworld-v1-688bdcf9f8-sx975 2/2 Running 0 42s pod/helloworld-v2-84d5d87fff-2qchg 2/2 Running 0 27s NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/helloworld ClusterIP 10.43.201.252 <none> 80/TCP 7s NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/helloworld-v1 1/1 1 1 42s deployment.apps/helloworld-v2 1/1 1 1 27s NAME DESIRED CURRENT READY AGE replicaset.apps/helloworld-v1-688bdcf9f8 1 1 1 42s replicaset.apps/helloworld-v2-84d5d87fff 1 1 1 27s ``` ## 建立 gateway ``` $ echo 'apiVersion: networking.istio.io/v1beta1 kind: Gateway metadata: name: canary-gateway namespace: test spec: selector: istio: ingressgateway servers: - port: name: http number: 80 # ingress-gateway port num protocol: HTTP hosts: - "*"' | kubectl apply -f - ``` ``` $ kubectl -n test get gateway.networking NAME AGE canary-gateway 18s ``` ## 建立 VirtualService * 設計 90% 的流量到 v1，10% 的流量到 v2 ``` $ echo 'apiVersion: networking.istio.io/v1alpha3 kind: VirtualService metadata: name: helloworld namespace: test spec: gateways: - canary-gateway hosts: - "*" http: - route: - destination: host: helloworld.test.svc.cluster.local subset: v1 weight: 90 - destination: host: helloworld.test.svc.cluster.local subset: v2 weight: 10' | kubectl apply -f - ``` ``` $ kubectl -n test get virtualService NAME GATEWAYS HOSTS AGE helloworld ["canary-gateway"] ["*"] 38s ``` ## 建立 DestinationRule * `DestinationRule` 是 Istio 裡用來定義服務後端子集（subsets）與連線策略的設定，讓你能根據標籤（labels）將流量分配到不同版本的 Pod。 ``` $ echo 'apiVersion: networking.istio.io/v1alpha3 kind: DestinationRule metadata: name: helloworld namespace: test spec: host: helloworld.test.svc.cluster.local subsets: - name: v1 labels: version: v1 - name: v2 labels: version: v2' | kubectl apply -f - ``` ``` $ kubectl -n test get destinationRule NAME HOST AGE helloworld helloworld.test.svc.cluster.local 4s ``` ## 檢視流量 * 使用 `istio-ingressgateway` service 的 nodeport 方式 curl，90% 流量會到 v1，10% 流量會到 v2 ``` $ kubectl -n istio-system get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE istio-ingressgateway NodePort 10.98.0.146 <none> 80:30962/TCP,443:31894/TCP 69s istiod ClusterIP 10.98.0.182 <none> 15010/TCP,15012/TCP,443/TCP,15014/TCP 18m kiali NodePort 10.98.0.22 <none> 20001/TCP,9090/TCP 16m ``` ``` $ while true; do curl -w "\n" http://10.10.7.30:30962/hostname; sleep 0.1; done {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v1-6969c997d7-5fdjb"} {"message":"helloworld-v2-6577fff95b-lqq9j"} {"message":"helloworld-v2-6577fff95b-lqq9j"} ``` ## UI 管理進到 kiali UI 可以看到流量管理 ![image](https://hackmd.io/_uploads/HkaBLHZVle.png) 進到 jaeger UI 可以看到 trace 流量 ![image](https://hackmd.io/_uploads/B1GVfNbVee.png) 可以看到連進服務花了多少時間 ![image](https://hackmd.io/_uploads/HJx5G4-Nlx.png)