---
# System prepended metadata

title: Chainsaw

---

# How to Perform Efficient E2E Testing with Chainsaw

![](https://hackmd.io/_uploads/ByXpMiIG6.png)

## Introduction

Chainsaw is a powerful tool that enables efficient end-to-end (E2E) testing, making it an essential asset for software developers. Built to be stronger and more cohesive than kuttl, Chainsaw offers a superior suite of features for Kubernetes (k8s) operators and objects. In this blog post, we will explore how to leverage Chainsaw for efficient E2E testing and enhance your overall testing workflow. Let's dive in!

## Table of Contents

- [Section 1: Installing Chainsaw](#section-1-installing-chainsaw)
  - [1.1 Downloading Chainsaw](#11-downloading-chainsaw)
  - [1.2 Installing Chainsaw](#12-installing-chainsaw)
- [Section 2: Writing Configurations](#section-2-writing-configurations)
  - [2.1 Understanding Chainsaw Configurations](#21-understanding-chainsaw-configurations)
  - [2.2 Writing Chainsaw Configurations](#22-writing-chainsaw-configurations)
- [Section 3: Creating Tests](#section-3-creating-tests)
  - [3.1 Understanding Chainsaw Tests](#31-understanding-chainsaw-tests)
  - [3.2 Creating Chainsaw Tests](#32-creating-chainsaw-tests)
- [Section 4: Running Tests](#section-4-running-tests)
  - [4.1 Preparing for Test Execution](#41-preparing-for-test-execution)
  - [4.2 Executing Tests in Chainsaw](#42-executing-tests-in-chainsaw)
- [Conclusion](#conclusion)


### Section 1: Installing Chainsaw

#### 1.1 Downloading Chainsaw

To get started with Chainsaw, you'll first need to download the appropriate release for your operating system and architecture. The releases are available on the official GitHub page at the following link: [Release](https://github.com/kyverno/chainsaw/releases)

Here's a breakdown of the available releases:

- **For macOS (Darwin) Users**:
  - Intel-based Macs: `chainsaw_darwin_amd64.tar.gz`
  - Apple Silicon Macs: `chainsaw_darwin_arm64.tar.gz`

- **For Linux Users**:
  - 32-bit: `chainsaw_linux_386.tar.gz`
  - 64-bit (AMD): `chainsaw_linux_amd64.tar.gz`
  - 64-bit (ARM): `chainsaw_linux_arm64.tar.gz`
  
> *Each release also comes with associated .pem, .sbom, .sbom.pem, and .sig files. These are used for verification and security purposes. If you're interested in verifying the integrity of your download, you can use these files. However, for the purpose of this guide, we'll focus on the main .tar.gz files.*

#### 1.2 Installing Chainsaw

Once you've downloaded the appropriate .tar.gz file for your system, you can proceed with the installation. Here's a step-by-step guide:

* **Extract the Tarball**:
   ```bash
   tar -xzf chainsaw_<your_version_and_architecture>.tar.gz
   ```
* **Move the Extracted Binary**:
    ```bash
    sudo mv chainsaw /usr/local/bin/
    ```

* **Verify the Installation**:
     ```bash 
     chainsaw --version
    ```
 
### Section 2: Writing Configurations

#### 2.1 Understanding Chainsaw Configurations

Chainsaw is a robust tool for end-to-end (e2e) testing in Kubernetes environments. It offers a flexible configuration system that allows users to define their testing parameters and conditions. The configuration can be set up in two primary ways:

* **Configuration File**: This is a structured file where you can define all your testing parameters.
* **Command-line Flags**: These are options you can pass directly when running Chainsaw commands. If both a configuration file and command-line flags are provided, the flags will override the settings in the configuration file.

Chainsaw follows a hierarchy in loading its configurations:

1. **User-specified Configuration**: If you provide a configuration file explicitly using a command-line flag.
2. **Default Configuration File**: If no configuration is specified, Chainsaw will search for a default file named .chainsaw.yaml in the current directory.
3. **Internal Default Configuration**: If neither of the above is available, Chainsaw will resort to a default configuration embedded within its binary.

To specify a custom configuration, use the following command:

```bash
chainsaw test --config path/to/your/config.yaml
```
#### 2.2 Writing Chainsaw Configurations

When writing a configuration for Chainsaw, you'll be working with a YAML file. Here's a basic example of what this might look like:

```yaml
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Configuration
metadata:
  name: custom-config
spec:
  timeout: 45s
  skipDelete: false
  failFast: true
  parallel: 4
  # ... other configurations
```

The full structure of the configuration file, including all possible fields and their descriptions, can be found in the [Configuration API reference](https://kyverno.github.io/chainsaw/apis/chainsaw.v1alpha1/#chainsaw-kyverno-io-v1alpha1-Configuration).

If you don't provide a custom configuration, Chainsaw will use its default:

```yaml
apiVersion: chainsaw.kyverno.io/v1alpha1
kind: Configuration
metadata:
  name: default
spec: {}
```

You can also override specific configurations using command-line flags, even after loading a configuration file. For instance:

```bash
chainsaw test --config path/to/your/config.yaml --timeout 45s --parallel 4
```

In this example, even if the configuration file specifies different values for `timeout` and `parallel`, the command-line flags will take precedence.

For a comprehensive list of all supported flags and their descriptions, refer to the [Chainsaw test command reference](https://kyverno.github.io/chainsaw/commands/chainsaw_test/#options).


### Section 3: Creating Tests

#### 3.1 Understanding Chainsaw Tests

In Chainsaw, a test is essentially an ordered sequence of test steps. These steps are executed sequentially, and if any step fails, the entire test is considered failed. Each test step can consist of multiple operations, such as creating, updating, or deleting resources in a Kubernetes cluster, or asserting that certain conditions are met.

Chainsaw supports three primary test definition mechanisms:

- **Manifests based syntax**: This is a straightforward method where you provide Kubernetes resource manifests. Chainsaw uses these manifests to create, update, or assert expectations against a cluster.
- **TestSteps based syntax**: A more verbose method that offers flexibility in defining each test step. It allows for additional configurations and can be combined with the Manifests based syntax. 
- **Test based syntax**: The most explicit method that provides a comprehensive overview of the test. It doesn't rely on file naming conventions and offers flexibility at both the test and test step levels.


#### 3.2 Creating Chainsaw Tests

Creating tests in Chainsaw requires a clear understanding of the test definition mechanisms. Here's a detailed guide for each syntax:

1. **Using Manifests based syntax**

   **File Naming**: Files should follow the convention `<step index>-<name|assert|error>.yaml`. For instance, `00-configmap.yaml` for resource creation, `01-assert.yaml` for assertions, and `02-error.yaml` for expected errors.
   
   **Example**:
   ```yaml
   # 00-configmap.yaml
   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: chainsaw-example
   data:
     key: value
    ```
2. **Using TestSteps based syntax**

    **Define a TestStep**: Create a YAML file that defines a `TestStep` resource. This resource should specify operations like `Apply`, `Assert`, `Delete`, and `Error`.
    
   **Example**:
    ```yaml!
    # 01-test-step.yaml
    apiVersion: chainsaw.kyverno.io/v1alpha1
    kind: TestStep
    metadata:
      name: apply-configmap-step
    spec:
      apply:
      - file: /resources/configmap.yaml
    ```
> You can combine TestStep resources with raw Kubernetes manifests. For instance, a TestStep might apply a resource, while a separate manifest file makes assertions about that resource.

   **Example**
   ```yaml
   # 02-assert.yaml
    apiVersion: v1
    kind: ConfigMap
    metadata:
      name: chainsaw-example
    data:
      key: value
   ```
3. **Using Test based syntax**

**Define a Test**: Start by creating a YAML file that defines a Test resource. This resource will have a spec section that outlines the entire test.

   **Example**
   ```yaml
    # chainsaw-test.yaml
    apiVersion: chainsaw.kyverno.io/v1alpha1
    kind: Test
    metadata:
      name: full-test-example
    spec:
      timeout: 10s
      steps:
      - spec:
          apply:
          - file: /resources/configmap.yaml
      - spec:
          assert:
          - file: /resources/configmap-assert.yaml
   ```
> Chainsaw processes the test by executing each step in sequence, ensuring that your Kubernetes environment meets the defined conditions.


### Section 4: Running Tests

#### 4.1 Preparing for Test Execution

Before executing your tests, ensure that:

1. **Kubernetes Cluster**: You have access to a Kubernetes cluster where the tests will run. This could be a local cluster (like Minikube or kind) or a remote one.
2. **Kubeconfig**: Your `kubeconfig` is correctly set up to point to the desired cluster. Chainsaw will use the current context from your `kubeconfig` to interact with the cluster.
3. **Test Files**: All your test files, whether they are written using Manifests based, `TestSteps` based, or `Test` based syntax, are organized and accessible.

#### 4.2 Executing Tests in Chainsaw

Once you're set up, running tests in Chainsaw is straightforward:

1. **Navigate to the Test Directory**: 
   ```bash
   cd path/to/your/test/directory
   ```
   
2. **Run the Tests**:
   ```bash
   chainsaw test
   ```
> This command will execute all tests in the current directory.

3. **View the Results**: Chainsaw will display the results in the terminal. It provides a summary of passed, failed, and skipped tests. For any failed tests, Chainsaw will display detailed error messages to help diagnose the issue.

The output will resemble:
```bash!
Loading default configuration...
...
Running tests...
=== RUN   quick-start
...
--- PASS: quick-start (5.22s)
PASS
Tests Summary...
- Passed  tests: 3
- Failed  tests: 1
- Skipped tests: 2
Done.
```


### Conclusion

Chainsaw provides a robust and flexible framework for end-to-end testing in Kubernetes environments. By understanding its configuration and test definition mechanisms, developers can create comprehensive test suites that ensure their Kubernetes applications and configurations are functioning as expected.