# SOS: Simplifying Operators Substantially

Recently, our team went to Cloud Native Rejekts '22 in Valencia, Spain to present on our [L5 Operator Demo](https://github.com/opdev/l5-operator-demo) project. From levels one through five, the majority of Kubernetes Operators settle in the mid range at level three. We wanted to change that with our L5 Operator Demo project which exemplfies what a basic level five Operator looks like. Unfortunately, before we could even begin to advocate the importance of Operator levels and higher capabilities, we had already hit a roadblock with this pressing question, "Why Operators?".

Before we can answer that question or dive into the levels of an Operator it's important to go over some foundational knowledge. By reading this article we hope to provided you with the who, what, when, where, why, and how (5W & 1H) of Operators and we'll simplify Operators substantially so that everyone can get a high-level understanding of what "Operators" are and why we believe they are so valuable in the Kubernetes, open source, and tech communities overall.

# 5W & 1H of Operators

## Who created Operators?
The term "Operator" was first mentioned in an [article](https://cloud.redhat.com/blog/introducing-operators-putting-operational-knowledge-into-software) written by Brandon Philips, the CTO of CoreOS, in 2016. As a team of Site Reliability Engineers who develop software specifically for application domains, his team had been working in the Kubernetes community designing and implementing a concept that would reliably create, configure, and manage complex application on top of Kubernetes. In his article he named this new class of software "Operators".


## What are Operators?

So, what are Operators? According to Philips, "An Operator is an application-specific controller that extends the Kubernetes API to create, configure, and manage instances of complex stateful applications on behalf of a Kubernetes user." This isn't an entirely new idea but an extension of existing Kubernetes resource and controller concepts with the addition of application-specific knowledge.

Operators contain three major components: *controller, resource*, and *knowledge*.


### 1. Controller
A controller is a control loop that watches the state of your cluster and moves your current cluster state closer to the desired state. Some examples of controllers that are commonly used in Kubernetes are Deployment controller, DaemonSet controller, and ReplicaSet controller.

### 2. Resource
A resouce, or more specifically a custom resource, is an endpoint in the Kubernetes API that stores a collection of API objects of a certain kind. Some examples of resources in Kubernetes are Pods, Secrets, and ConfigMaps.

In regards to Operators, a Custom Resource (CR) is a custom kind users can create that allows them to store their own API Objects while the API server handles the lifecycle of the CR. Alone, a CR stores and retrieves structured data. It is only an instance of the Custom Resource Definition (CRD), so a CRD must be defined before an instance of the CR can be created. Below is an example of what CRDs look like:


### 3. Knowledge
Knowledge is domain or application specific, which is often learned from users or administrators rather than developers. Some examples of application specific knowledge that can be implemented in an Operator are installation, scaling, self-healing, backup & restore, updates, clean-up, resiliency, and observability.

The L5 Operator implements knowledge in its installation stage. The application requires the PostgreSQL database to be seeded before the application pods are instantiated, so in our Operator we created a job with the required information to seed our database pods before the application pods are created. This ensures that the database exists before the application tries to access it.

### Operator Capability

Operators consist of different levels of capability. The [Operator capability levels](https://sdk.operatorframework.io/docs/overview/operator-capabilities/) infographic below displays the range of knowledge each level is capable of. It also shows which level each Operator tooling can achieve. Helm can create Operators up to a level three caliber, while Ansible and Go can create complete auto-pilot Operators that fully manage applications.


## Why are Operators valuable?

Operators are valuable because they build an ecosystem of software that can be easy, safe, and reliable to use and operate as a cloud service. They ease the workload that Kubernetes users would otherwise have to do manually. Operators provide a means to simply install and work across applications and software. They are low-touch, remotely managed, with one-click updates and provide consistency and stability in the world of cloud services and containerization bringing functionality were its wanted.

By providing this knowledge to more developers, we can continue enhancing the average capability level of Operators in the ecosystem and enable enterprises and developers to create easier, safer, and more reliable cloud native applications, which will in turn lead to even more innovation in the space and more transitions to cloud native solutions. Since the Operator Framework, an Operator toolkit, is open source software, Operators rely upon the community to create and further develop them into the next generation of application management and automation. The more we all work together, the better Operators become.

## Where are Operators currently being used?

Operators are being used everywhere. OperatorHub.io is a community library of existing Operators, with listings from Amazon, Dell, Prometheus, PostgreSQL, Grafana, MongoDB, Couchbase, GitLab, Quay, Redis, Red Hat, etc., they all have Operators available in OperatorHub.


## When should we use Operators?

Of course, as amazing as they are and as useful as they are, Operators aren't the answer to all applications. Operators are difficult and can be tedious, depending on your needs. They take time and expert knowledge of your application and can be more work than you bargin for if your application doesn't quite need the help. So here are some guiding questions as to whether or not your applicaiton can benefit from an Operator:

* Is your application a *Stateful Application?* Operators are at their most useful when helping to manage stateful applications in a Kubernetes setup.
* Does your application contain *application specific tasks* that could benefit from managed automation that Kubernetes doesn't already fulfill?
* Could your application benefit from being more *scalable, repeatable,* and *standardized*?

## How do we create our own Operator?

Although the learning curve can be steep, there are tools and communities that make onboarding straight forward. [Operator Framework](https://operatorframework.io/) is an open source toolkit that manages Operators in an effective, automated, and scalable way. It consists of three major toolsets–[Operator SDK](https://sdk.operatorframework.io/), [Operator Lifecycle Manager (OLM)](https://olm.operatorframework.io/), and OperatorHub.io

## Operator Framework

### Operator SDK

Operator SDK provides tools to build, test, and package Operators. It is a framework that uses the controller-runtime library to make writing Operators easier by providing high level APIs to write operational logic intuitively, tools for scaffolding and code generation to bootstrap a new project fast, and extensions to cover common Operator use cases.

How to install Operator-SDK:
* Mac: `brew install operator-sdk`
* Linux/Windows:
1. `git clone https://github.com/operator-framework/operator-sdk`
2. `cd operator-sdk`
3. `git checkout master`
4. `make install`

### Operator Lifecycle Manager(OLM)
OLM extends Kubernetes to provide a declarative way to install, manage, and upgrade Operators on a cluster. OLM features include over-the-air updates and catalogs which keep operators up tp date automatically. It has a dependency model that allows Operators to express dependencies on the platform and on other Operators. OLM makes Operators and their services available for cluster users to select and install. It provides cluster stability by preventing conflicting Operators owning the same API's.

#### OLM Architecture

OLM itself runs on operators, specifically two operators named as follows:
1. olm-operator
2. catalog-operator

Each of these Operators manage the CRD's that are the basis of the OLM framework. Following is the list of CRD's managed by olm-operator and catalog-operator:
1. ClusterServiceVersion
2. InstallPlan
3. CatalogSource
4. Subscription
5. OperatorGroup

#### OLM Operator
OLM Operator deploys the applications defined by a ClusterServiceVersion (CSV) after the required resources specified in the CSV are present in the cluster. A CSV is a YAML manifest created from Operator metadata that assists OLM in running the Operator in a cluster. Since the OLM Operator is not concerned with the creation of the resources, a user can choose to either create these resources manually using CLI or opt for creating using Catalog Operator. OLM Operator is configured in such a way that it watches all namespaces. It watches for CSV and checks that all requirements are met. If so, it runs the install strategy for the CSV.

#### Catalog Operator
The Catalog Operator resolves and installs CSV's and the required resources they specify. Catalog Operator watches for CatalogSource and Subscriptions and creates InstallPlans based on them. CatalogSource is the collection of metadata defined in CSV. Subscription describes which channel of an Operator package to subscribe to, and whether to perform updates automatically or manually. It defines the name and namespace of the Operator. Catalog Operator also watches for resolved and unresolved InstallPlans created by the user and creates all of the discovered resources for it.

### OperatorHub.io
OperatorHub.io is a home for the Kubernetes community to share Operators. OperatorHub.io was launched and created by RedHat, alongside Amazon, Microsoft, and Google as the inital supporters. OperatorHub.io aims to be the central location where users can find a wide array of community built Operators. This library was made for both Operator developers and end users. OperatorHub.io provides developers a place to publish and update their work and defines criterias and requirements for publication. Operators on OperatorHub.io are standardized to provide end users with basic documentation and easy access to deploy and manage Operators from their own Kubernetes cluster. OperatorHub.io is the front-end for the [Community Operator Repository](https://github.com/k8s-operatorhub/community-operators) on GitHub, so to get an Operator published to OperatorHub.io, developers need only to package their Operator via OLM and submit a Pull Request to the [repository](https://github.com/k8s-operatorhub/community-operators) to be reviewed. For more information on how you can contribute, check out their contribution [site](https://operatorhub.io/contribute).

### RedHat Openshift Container Platform


RedHat Openshift Container Platform(RHOCP) is a consistent hybrid cloud foundation for building and scaling containarized applications. This platform helps developers build applications offering automated installation, upgrades, and life cycle management throughout the container stack. It uses microservice based architecture with smaller, decoupled units that work together. RedHat Openshift Container Platform runs the Red Hat Enterprise Linux (RHEL) operating system and functions as a set of Docker-based application containers managed with Kubernetes orchestration. This platform is mainly used in hybrid cloud scenarios.

---

### Scaffolding Your Own Operator

In this quick demo, we'll be following this [tutorial](https://sdk.operatorframework.io/docs/building-operators/golang/tutorial/) to get you set up and create your own operator

1. ```
mkdir -p $HOME/projects/memcached-operator
cd $HOME/projects/memcached-operator

    
2. ```
    operator-sdk init --domain example.com --repo github.com/example/memcached-operator

3. ```
operator-sdk create api --group cache --version v1alpha1 --kind Memcached --resource --controller

---

# Resources

## Code

* [L5 Operator](https://github.com/makon57/l5-operator-demo)
    * Pull code base and branches level_1-5
    * Install GitLens extension on VSCode
    * Toggle VSCode to Source Control and open tab Search & Compare
    * Add Compare References... (will open dropdown to select branches you want to compare, select 2 options)

## Labs

* [Operator Workshop Labs](http://workshop.coreostrain.me/support/)
* [Building Operators on OpenShift Labs Exercises](https://developers.redhat.com/learn/openshift/operators)
* [Learn Golang w/ Exercism](https://exercism.org/tracks/go)

## Articles:

* [CNCF - Kubernetes Operators: What are they? Some examples](https://www.cncf.io/blog/2022/06/15/kubernetes-operators-what-are-they-some-examples/)
* [What is a Kubernetes Operator?](https://www.redhat.com/en/topics/containers/what-is-a-kubernetes-operator)
* [What is the Kubernetes API?](https://www.redhat.com/en/topics/containers/what-is-the-kubernetes-API#:~:text=The%20Kubernetes%20API%20is%20the,valid%20and%20then%20processes%20it.)

## Slide Deck

* [SOS Google Slide Deck](https://docs.google.com/presentation/d/1DAWDfKC4bUCEioggyGIOZh0IRVr2LGny84asih1ZWPc/view?usp=sharing)