# Cityeyes to k8s Kevin(5f) A40503 0X000, ITRI ICL --- ## First idea --- ## Containerization --- ## Container-centric management --- <!-- .slide: data-transition="zoom" data-background="#FFF" -->  --- ## Why do I need Kubernetes and what can it do? ---- Kubernetes has a number of features. It can be thought of as: - <span>a container platform<!-- .element: class="fragment" data-fragment-index="1" --></span> - <span>a microservices platform<!-- .element: class="fragment" data-fragment-index="2" --></span> - <span>a portable cloud platform and a lot more.<!-- .element: class="fragment" data-fragment-index="3" --></span> ---- <span> **Platform as a Service (PaaS)** <!-- .element: class="fragment" data-fragment-index="1" --></span> <span> **Infrastructure as a Service (IaaS)** <!-- .element: class="fragment" data-fragment-index="2" --></span> --- <!-- .slide: data-background="#FFF" --> ## Concepts about Kubernetes ---- ## Cluster ---- <!-- .slide: data-background="#FFF" -->  ---- ## Persistent Volumes ---- <!-- .slide: data-background="#FFF" -->  ---- ## Containers ---- <!-- .slide: data-background="#FFF" -->  ---- ## Pods ---- <!-- .slide: data-background="#FFF" -->  ---- ## Deployments ---- <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-background="#FFF" -->  ---- ### How to deploy on specific node ? ---- <!-- .slide: data-background="#FFF" -->  --- ## Kubernetes architecture ---- <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-background="#FFF" -->  --- ## Deeplook ---- ## Components ---- <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-transition="fade-in convex-out" data-background="#FFF" -->  --- ## Try to simplify <!-- .slide: data-background="https://i.imgflip.com/s00vb.jpg" --> ---- <!-- .slide: data-transition="fade-in convex-out" data-background="#FFF" -->  ---- <!-- .slide: data-transition="fade-in convex-out" data-background="#FFF" -->  ---- <!-- .slide: data-transition="fade-in convex-out" data-background="#FFF" -->  ----   ---- <!-- .slide: data-transition="fade-in convex-out" data-background="#FFF" -->  ----  ***    ---- ## Build small containers ---- <!-- .slide: data-transition="fade-in convex-out" data-background="#FFF" -->   ---- [Kubernetes best practices: How and why to build small container images](https://www.youtube.com/watch?v=wGz_cbtCiEA) ---- ### Monolithic ![image alt ](https://cdn-1.wp.nginx.com/wp-content/uploads/2016/04/Richardson-microservices-part1-1_monolithic-architecture.png "Monolithic" =530x580) ---- <!-- .slide: data-transition="zoom" --> ### Microservices ![image alt ](https://cdn-1.wp.nginx.com/wp-content/uploads/2016/04/Richardson-microservices-part1-2_microservices-architecture.png "Microservices" =530x580) --- ## Deploy <!-- .slide: data-background="https://davidjellison.files.wordpress.com/2013/10/deploy-button.jpg" --> ---- ### Deploy ways - <span> Deploy from file(yaml) <!-- .element: class="fragment" data-fragment-index="1" --></span> - <span> Deploy from package manager <!-- .element: class="fragment" data-fragment-index="2" --></span> ---- #### Deploy from file(yaml) ---- nginx-deployment.yaml ```yaml apiVersion: apps/v1 kind: Deployment metadata: name: nginx-deployment labels: app: nginx spec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.7.9 ports: - containerPort: 80 ``` ---- ```shell $ kubectl create -f nginx-deployment.yaml ``` ---- ```shell NAME READY STATUS RESTARTS AGE pod/nginx-deployment-787d795c5b-9sb49 1/1 Running 0 8d NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/nginx NodePort 10.104.140.140 <none> 80:32000/TCP 8d NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE deployment.apps/nginx-deployment 1 1 1 1 8d NAME DESIRED CURRENT READY AGE replicaset.apps/nginx-deployment-787d795c5b 1 1 1 8d ``` ---- #### Deploy from package manager ---- <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-transition="zoom" data-background="#FFF" -->  ---- <!-- .slide: data-background="#FFF" -->  ---- ![image alt ](https://pbs.twimg.com/media/DYvzE2CWkAMzNm3.jpg "kubeapps hub" =560x600) ----  ----  ---- ![image alt ](https://i.imgur.com/whJ2Pd0.png "value.yaml" =650x600) ---- ```shell $ helm install --name jenkins stable/jenkins -f values.yaml ``` ---- ```shell NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE jenkins LoadBalancer 10.106.218.56 <pending> 8080:32707/TCP 2h ``` ----  --- ## <font color="green">Configure</font> <!-- .slide: data-background="http://exploreflask.com/en/latest/_images/configuration.png" --> ---- ## Configmap/Sercet ---- <!-- .slide: data-background="#FFF" -->  ---- ## Secret Example ---- ### pgadmin-secret.yaml ```yaml apiVersion: v1 kind: Secret metadata: name: pgadmin-secret namespace: default type: Opaque data: username: YWRtaW5AYWRtaW4uY29t # admin@admin.com | base64 password: YWRtaW4= # admin | base64 ``` ---- ```shell $ kubectl create -f pgadmin-secret.yaml ``` ---- ### pgadmin-deployment.yaml ```yaml ... spec: containers: - name: pgadmin image: dpage/pgadmin4 env: - name: PGADMIN_DEFAULT_EMAIL valueFrom: secretKeyRef: name: pgadmin-secret key: username - name: PGADMIN_DEFAULT_PASSWORD valueFrom: secretKeyRef: name: pgadmin-secret key: password ports: - containerPort: 80 ... ``` ---- ## Configmap Example ---- ### config.properties ### (engine - orchestrator) ```shell LogServiceURL=http\://paas\:8080/PaaS/service/REST PaasServiceURL=http\://paas\:8080/PaaS/service/REST JobServiceURL=null ##RabbitMQ Connection RabbitMQServiceHost=mq-rabbitmq RabbitMQServicePort=5672 RabbitQMServiceVHost=/ RabbitMQServiceUsername=snake_eyes RabbitMQServicePassword=123456 ##RabbitMQ Priority Settings RabbitMQAssignQueue=-1 RabbitMQPriorityEnable=TRUE PriorityRangeBound_1st=20 PriorityRangeBound_2nd=60 PriorityRangeBound_top=140 CNName=classify DefaultTaskEnginePath= LogExpiryDay=10 ... ``` ---- ```shell $ kubectl create configmap engine-classify --from-file=config.properties ``` ---- ### engine_classify.yaml ```shell ... volumes: - name: config configMap: name: engine-classify ... ``` ```shell ... volumeMounts: - name: config mountPath: /root/config.properties subPath: config.properties ... ``` ---- ### ! <!-- .slide: data-background="https://i.imgflip.com/4j4d5.jpg" -->  ---- ### engine_classify.yaml <!-- .slide: data-background="https://i.imgflip.com/quwu6.jpg" --> ```shell ... volumes: - name: config configMap: name: engine-classify ... ``` ```shell ... volumeMounts: - name: config mountPath: /root/tmp_config ... ``` ```shell ... command: ["/bin/bash","-c"] args: - ln -s -f /root/tmp_config/config.properties /root/config.properties; ... ``` --- ## Volume <!-- .slide: data-background="https://i.imgflip.com/2f9cmh.jpg" --> ---- <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-background="#000" -->  ---- <!-- .slide: data-background="#000" -->  ---- ### Types of Persistent Volumes ``` - GCEPersistentDisk - AWSElasticBlockStore - AzureFile - AzureDisk - FC (Fibre Channel) - Flexvolume - Flocker - NFS - iSCSI - RBD (Ceph Block Device) - CephFS - Cinder (OpenStack block storage) - Glusterfs - VsphereVolume - Quobyte Volumes - HostPath (Single node testing only – local storage is not supported in any way and WILL NOT WORK in a multi-node cluster) - Portworx Volumes - ScaleIO Volumes - StorageOS ``` ---- ### One node <!-- .slide: data-background="#FFF" -->  ---- ### Cross node <!-- .slide: data-background="#FFF" -->  ---- ### Cross node solution <!-- .slide: data-background="#FFF" -->  ---- ### NFS Persistent Volumes <!-- .slide: data-background="#FFF" -->  --- <!-- .slide: data-background="https://usan.com/wp-content/uploads/2013/09/customer-self-service.jpg" --> ---- ### Service Type ---- ### ClusterIP  ---- ### ClusterIP Example ```yaml apiVersion: v1 kind: Service metadata: name: nfs-server spec: clusterIP: 10.100.95.27 ports: - name: nfs port: 2049 - name: mountd port: 20048 - name: rpcbind port: 111 selector: role: nfs-server ``` ---- ### NodePort  ---- ### NodePort Example ```yaml apiVersion: v1 kind: Service metadata: name: deeplook labels: app: deeplook spec: type: NodePort ports: - port: 8080 nodePort: 32010 selector: app: deeplook ``` ---- ### LoadBalancer  ---- <!-- .slide: data-background="#FFF" --> | | | |-|-| ||| ---- ### Ingress  ---- ### Ingress Example ```yaml apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx nginx.ingress.kubernetes.io/ssl-redirect: "false" nginx.ingress.kubernetes.io/force-ssl-redirect: "false" nginx.ingress.kubernetes.io/add-base-url: "true" nginx.ingress.kubernetes.io/rewrite-target: / name: manage namespace: default spec: rules: - http: paths: - path: /media/ backend: serviceName: nginx servicePort: 80 - path: /map/ backend: serviceName: map servicePort: 80 - path: /rabbitmq backend: serviceName: mq-rabbitmq servicePort: 15672 ``` ---- [Kubernetes NodePort vs LoadBalancer vs Ingress? When should I use what?](https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0) --- ## Log <!-- .slide: data-background="https://www.wur.nl/upload_mm/9/2/e/d5625cbd-9d40-4bf3-8f78-e2a352af5e9d_shutterstock_logs_a9e60c17_490x330.jpg" --> ---- ### docker log ---- ```shell $ kubectl logs nginx-deployment-86678d658d-jmbmb ``` ---- ## EFK & EFK <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-background="#FFF" --> |Before Fluentd|After Fluentd| |:-:|:-:| || | ---- ### Simplified schema of an EFK <!-- .slide: data-background="#FFF" -->  ----  ----  ---- <span>docker exec -it <container> sh<!-- .element: class="fragment" data-fragment-index="1" --></span> <span> => tail -n -f <log files><!-- .element: class="fragment" data-fragment-index="2" --></span> <span> <!-- .element: class="fragment" data-fragment-index="3" --> </span> ----  ----  ---- ### Streaming sidecar container <!-- .slide: data-background="http://i.giphy.com/90F8aUepslB84.gif" -->  ---- {%pdf https://static.googleusercontent.com/media/research.google.com/zh-TW//pubs/archive/45406.pdf %} ---- ```shell $ kubectl logs --tail=55 -lapp=engine-classify -c engine-classify-agent ``` ----  ----  --- ## Deploy Deeplook <!-- .slide: data-background="https://i.imgflip.com/woiri.jpg" --> ---- ### Clone project ```shell $ git clone https://gitlab.com/Aquamars/Deeplook_k8s.git ``` ---- ### Follow: [Deeplook_k8s Deploy](https://gitlab.com/Aquamars/Deeplook_k8s/tree/master/Deploy) --- ## Initial data ---- <!-- .slide: data-background="#FFF" -->  --- ## Deploy Cityeyes & Paas ---- ### Deploy from code ---- <!-- .slide: data-background="#FFF" -->  ---- <!-- .slide: data-background="#FFF" -->  ---- My project on private repository...  ---- ## Use Token <!-- .slide: data-background="#FFF" -->  ---- ```shell $ git clone --single-branch -b v1.1.0 https://0123456789012345678901234567890123456789@github.com/ITRI-ICL-DivX/deeplook_cityeyes.git ``` ---- ### Container 1 (git clone project) ```yaml initContainers: - name: gitclone image: alpine/git command: ["sh","-c","rm -rf /git/* && git clone --single-branch -b v1.1.0 https://2e990ac7f19037aca1b3672ee443f597a23e2995@github.com/ITRI-ICL-DivX/deeplook_cityeyes.git && sleep 2"] volumeMounts: - name: git-repo mountPath: /git ``` ---- ### Container 2 (build project) ```yaml containers: - name: build-war image: maven:3.5.4-jdk-8-alpine command: ["sh","-c"] args: - cd /usr/src/mymaven/deeplook_cityeyes; sed -i "s|\/hot||g" ./src/main/webapp/OpenLayers/GlobleVal.js; sh /usr/src/mymaven/deeplook_cityeyes/settingScript --paas_url paas --db_url cityeyes-postgresql/postgres --samba 140.96.29.86/media/ --map_url 140.96.29.86/map/styles/osm-bright --acl_url acl:8080; mvn clean package -nsu -Dmaven.test.skip=true; cp /usr/src/mymaven/deeplook_cityeyes/target/CityEyes.war /target; while :; do sleep 2073600; done; volumeMounts: - name: maven-repo mountPath: /root/.m2 - name: git-repo mountPath: /usr/src/mymaven - name: after-build mountPath: /target ``` ---- ### Container 3 (running system) ```yaml - name: cityeyes image: tomcat:8-jre8 env: - name: CATALINA_OPTS value: "-Duser.timezone=Asia/Taipei" volumeMounts: - name: after-build mountPath: /usr/local/tomcat/webapps - name: git-repo mountPath: /repo - name: media-root mountPath: /var/www/html/ ports: - containerPort: 8080 ``` ---- <!-- .slide: data-background="#FFF" -->  ----  --- # Deployment Strategies ---- ## Rolling-update <!-- .slide: data-background="https://www.cat-gifs.com/w3/Funny-Kitten-GIF-Playful-Kitty-rolling-in-a-fitness-carpet-enjoying-a-new-game-with-his-human.gif" --> ---- <!-- .slide: data-background="#FFF" -->  ---- ## Canary deployment <!-- .slide: data-background="https://i.imgflip.com/2cczo7.jpg" --> ---- <!-- .slide: data-background="#FFF" -->  ---- ## Blue-Green deployment <!-- .slide: data-background="https://58generalclass.files.wordpress.com/2017/08/img_0359.jpg" --> ---- <!-- .slide: data-background="#FFF" -->  --- # Can do more ---- ## Monitor <!-- .slide: data-background="#FFF" -->  ---- ## Auto Deploy <!-- .slide: data-background="#FFF" -->  ---- ## Image registry <!-- .slide: data-background="#FFF" -->  ---- ## Manager tensorflow <!-- .slide: data-background="#FFF" -->  ---- # Service Mesh  ---- # more and more... --- # DevOps ---- <!-- .slide: data-background="#FFF" -->  ----  --- ## References#1 [Explain the "size" and "virtual size"(issuecomment)](https://github.com/docker/docker.github.io/issues/1520#issuecomment-305179362) [Why do I need Kubernetes and what can it do?](https://kubernetes.io/docs/concepts/overview/what-is-kubernetes/#why-do-i-need-kubernetes-and-what-can-it-do) [Kubernetes Objects](https://kubernetes.io/docs/concepts/#kubernetes-objects) [Container size on disk](https://docs.docker.com/v17.09/engine/userguide/storagedriver/imagesandcontainers/#container-size-on-disk) [Introduction to microservices](https://www.nginx.com/blog/introduction-to-microservices/) [Draft-vs-gitkube-vs-helm-vs-ksonnet-vs-metaparticle-vs-skaffold](https://blog.hasura.io/draft-vs-gitkube-vs-helm-vs-ksonnet-vs-metaparticle-vs-skaffold-f5aa9561f948) [Helm](https://helm.sh/) [volumes-configmap](https://kubernetes.io/docs/concepts/storage/volumes/#configmap) [types-of-persistent-volumes](https://kubernetes.io/docs/concepts/storage/persistent-volumes/#types-of-persistent-volumes) ---- ## References#2 [Kubernetes Volumes](https://www.youtube.com/watch?v=VB7vI9OT-WQ&t=9s) [NFS Persistent Volumes with Kubernetes ](https://medium.com/platformer-blog/nfs-persistent-volumes-with-kubernetes-a-case-study-ce1ed6e2c266) [Kubernetes NodePort vs LoadBalancer vs Ingress? When should I use what?](https://medium.com/google-cloud/kubernetes-nodeport-vs-loadbalancer-vs-ingress-when-should-i-use-what-922f010849e0) [EFK stack? Like ELK?](https://blog.ptrk.io/how-to-deploy-an-efk-stack-to-kubernetes/) [using-a-sidecar-container-with-the-logging-agent](https://kubernetes.io/docs/concepts/cluster-administration/logging/#using-a-sidecar-container-with-the-logging-agent) [Design patterns for container-based distributed systems](https://static.googleusercontent.com/media/research.google.com/zh-TW//pubs/archive/45406.pdf) [kubernetes-best-practices](https://speakerdeck.com/thesandlord/kubernetes-best-practices) ---- ## References#3 [creating-a-personal-access-token-for-the-command-line](https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/) [Blue/Green Deployment, Rolling update and Canary Deployment](https://www.jianshu.com/p/022685baba7d) [Hahow blue green deployment](https://tech.hahow.in/%E6%B7%BA%E8%AB%87-hahow-%E8%97%8D%E7%B6%A0%E9%83%A8%E7%BD%B2-1c99d42336a9) [Deployment Strategies](https://container-solutions.com/deployment-strategies/) [BlueGreenDeployment](https://martinfowler.com/bliki/BlueGreenDeployment.html) [istio](https://github.com/istio/istio) [service-mesh-istio](http://www.inwinstack.com/zh/2018/01/12/service-mesh-istio/) [devops](https://www.smartsheet.com/devops) [Getting Your Arms Around DevOps](https://blogs.gartner.com/cameron_haight/2014/10/13/getting-your-arms-around-devops-devops-patterns-and-practices/) --- # The End <!-- .slide: data-background="https://i.makeagif.com/media/3-20-2015/9ttdrX.gif" -->