# Chaos Engineering

# LitmusChaos

## Introduction

**Litmus** is an open source cloud-native Chaos Engineering framework that provides custom APIs to orchestrate chaos on Kubernetes from ChaosHub.

Litmus applies a cloud-native approach to create and manage chaos. A **Chaos** is orchestrated using the following CRDs:

1. **ChaosEngine**: It is a resource that links itself to some Kubernetes applications.
2. **ChaosExperiment**: It is s resource that groups the configuration of chaos experiments. They are created by the Litmus operator when some experiments are invoked by ChaosEngine.
3. **ChaosResult**: A resource that contains the results of a ChaosExperiment..

## Installation

kubectl apply -f https://litmuschaos.github.io/litmus/litmus-operator-v1.6.0.yaml

## Configure ChaosExperiment

The Chaos experiments contain the actual chaos details. The experiments are grouped as Chaos Charts and are published on [Chaos Hub](https://hub.litmuschaos.io/).The **Chaos Hub** is a central hub where developers share their chaos experiments.

In this documentation, we use a Chaos Chart called *generic chaos experiments* .The *generic chaos experiments* contains some common Chaos Charts including `pod-delete`, `container-kill`, `pod-network-latency`.

kubectl apply -f https://hub.litmuschaos.io/api/chaos/1.6.0?file=charts/generic/experiments.yaml -n elk

## RBAC setting

Example `rbac.yaml`:

---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: container-kill-sa
  namespace: nginx
  labels:
    name: container-kill-sa
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: container-kill-sa
  namespace: nginx
  labels:
    name: container-kill-sa
rules:
- apiGroups: ["","litmuschaos.io","batch","apps"]
  resources: ["pods","jobs","pods/exec","pods/log","events","chaosengines","chaosexperiments","chaosresults"]
  verbs: ["create","list","get","patch","update","delete"]
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: container-kill-sa
  namespace: nginx
  labels:
    name: container-kill-sa
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: container-kill-sa
subjects:
- kind: ServiceAccount
  name: container-kill-sa
  namespace: nginx

create rbac account on the elk namespace:

kubectl create -f rbac.yaml -n elk

## Prepare ChaosEngine

ChaosEngine links the application to the ChaosExperiment.

Example yaml:

apiVersion: litmuschaos.io/v1alpha1
kind: ChaosEngine
metadata:
  name: elk-chaos
  namespace: elk
spec:
  # It can be true/false
  annotationCheck: 'true'
  # It can be active/stop
  engineState: 'active'
  #ex. values: ns1:name=percona,ns2:run=nginx
  auxiliaryAppInfo: ''
  appinfo:
    appns: 'elk'
    applabel: 'app=elk'
    appkind: 'deployment'
  chaosServiceAccount: container-kill-sa
  monitoring: false
  # It can be delete/retain
  jobCleanUpPolicy: 'delete'
  experiments:
    - name: container-kill
      spec:
        components:
          env:
            # specify the name of the container to be killed
            - name: TARGET_CONTAINER
              value: 'elk-with-ssh'

            # provide the chaos interval
            - name: CHAOS_INTERVAL
              value: '10'

            # provide the total chaos duration
            - name: TOTAL_CHAOS_DURATION
              value: '20'

            # For containerd image use: litmuschaos/container-kill-helper:latest
            - name: LIB_IMAGE
              value: 'gaiaadm/pumba:0.6.5'

            # It supports pumba and containerd
            - name: LIB
              value: 'pumba'

:::info
:bulb: Pumba is a chaos testing command line tool for Docker containers
:::

Run the following command to launch ChaosEngine

kubectl apply -f chaosengine.yaml

## Examine result

We can use the kubectl describe command to examine the status of the experiment. If the `status.verdict` is set to `Awaited`, then the experiment is currently in progress. Otherwise, the `status.verdict` will change to either `Pass` or `Fail` eventually.
Describe the ChaosResult CR to know the status of each experiment. The status.verdict is set to Awaited when the experiment is in progress, eventually changing to either Pass or Fail.

kubectl describe chaosresult nginx-chaos-container-kill -n elk

## Troubleshooting

We can use the following command to examine the log file of litmus:

kubectl logs -f <chaos-operator-(hash)-(hash)>-runner -n litmus