# Ratify policy for evaluation of container image vulnerability reports at admission

## Overview

### Problem/Motivation

Ratify is a verification engine that enables you to verify the security metadata of artifacts and deploy only those that comply with your policies. While Ratify supports the verification of the integrity and authenticity of container images, it’s important to note that trusted images are not necessarily free of vulnerabilities. Deploying vulnerable images poses a security risk.

### User stories

Alice, a security engineer at Contoso, needs to ensure that the container images running in K8s clusters are secure. She has installed OPA Gatekeeper and Ratify in the K8s clusters to configure admission policies that prevent deploying untrusted, vulnerable, and non-compliant container images. She has enforced the integrity policy to ensure that only trusted images are deployed. However, she has not found policies to prevent the deployment of vulnerable images. Therefore, she would like to have a policy that verifies vulnerability reports of container images.

Alice ensures that vulnerability reports are trusted by configuring the policy to validate their signatures. Addtiionally, she configures the freshness of vulnerability reports to prevent deploying images with outdated vulnerability reports.

Alice configures the policy to block images that have reported CVEs with different severities, such as Critical and High severities.

She is aware of VEX documents that can be used to verify the current status of vulnerabilities, so she would like the policy to evaluate VEX documents by default, provided VEX documents are available.

Addtionally, Alice configures the policy to prevent images containing specific CVEs that Contoso's company strictly denies.

Alice seeks flexible policy enforcement. In the production environment, she denies image deployment if policy evaluation fails. However, for testing and development environments, she configures the policy to allow image deployment with auditing logs even when policy evaluation fails.

Since Contoso’s security and compliance rules may change over time, and new vulnerabilities can be discovered over time. Alice wants to update the policy configuration in real-time to reflect the new rules.

### Goals/Non-Goals

Goals:
* Provide users with a built-in rego policy that prevents deploying vulnerable images.
* The policy should be parameterized so that users can set different values.
* Users can update the parameters in real-time.
* Support vulnerability reports in standard format, such as SARIF.
* Support VEX documents, like openVEX, cyclonedx VEX
* The configuration used for image signature verification is also used for signature verification.

Non-goals:

* Support non-standard vulnerability reports generated by different tools.
* Support non-rego policy.

### Proposal

The proposal aims to enhance Ratify by adding support for verifying new types of data, namely vulnerability reports and VEX documents. The vulnerability report should be in standard format, such as SARIF. A built-in rego policy will be provided to users, and they will only need to update policy parameters to achieve the goal of preventing vulnerable images.

### User Experience

#### Set up the policy for vulnerability report verification

The following describes the procedure of setting up policies for a fresh installation:

1. Install Gatekeeper.
1. Install Ratify with built-in policies.
1. Apply the policy of preventing deployment of vulnerable image.
1. Configure enforcement actions based on different environments
- For testing environment, set `enforcementAction` to `warn` or `dryrun` in Gatekeeper constraint. To apply the parameter, run the command `kubectl apply -f constraint_warn.yaml` or `kubectl apply -f constraint_dryrun.yaml`.
- For development environment, set `enforcementAction` to `warn` or `dryrun` in Gatekeeper constraint. To apply the parameter, run the command `kubectl apply -f constraint_warn.yaml` or `kubectl apply -f constraint_dryrun.yaml`.
- For production environment, set `enforcementAction` to `deny` in Gatekeeper constraint. To apply the parameter, run the command `kubectl apply -f constraint_deny.yaml`.
1. Pass values to parameters defined by the policy. Assuming these parameters are listed in file `policy_vulnerable_report_parameters.yaml`.
| Parameters | Description | Default |
| ----------------- | ---------------------------------------------------------------------------------------------------- | --------------------------------------- |
| signature.enabled | Whether validating signatures or not, if enabled, use the same signature verification from the image | true |
| maxReportAge | The maximum acceptable age (in hour) of the vulnerability report | 24 |
| cveDenyList | A list of CVEs that are NOT allowed | "" |
| severities | Block the images based on vulnerability severities, allwed values: crtical, high, medium, low | critical, high |

To pass the values to parameters, run the following command:

`kubectl apply -f policy_vulnerable_report_parameters.yaml`

#### Update policy parameters at real-time

Assuming policy parameters are configured in file `policy_vulnerable_report_parameters.yaml`. At runtime, users update this file with new values, and then run the following command:

`kubectl apply -f policy_vulnerable_report_parameters.yaml`

#### Workflow in Rego policy for vulnerability report verification

The following steps shows the workflow in the Rego policy.

1. Check whether the target image has vulnerability reports.
- If there are no vulnerability reports, whether the deployment of the image is denied or not depends on the enforcement actions.
- If there are vulnerability reports, continue to the next step.
1. Check the parameter `signature.enabled` for signature verification.
- If signature verification is required, validate signatures based on the `signature.type` and corresponding configuration.
- If verification fails, whether the deployment of the image is denied or not depends on the enforcement actions.
- If verification succeeds, continue to the next step.
- If signature verification is not enabled, continue to the next step.
1. Check whether the vulnerability report is older than the value of parameter `maxReportAge`.
- If it is older, whether the deployment of the image is denied or not depends on the enforcement actions.
- If it is not older, continue to the next step.
1. Check whether there are CVEs in the CVE deny list based on the value of parameter `cveDenyList`.
- If there is any CVE in the deny list, whether the deployment of the image is denied or not depends on the enforcement actions.
- If there are no CVEs in the deny list, continue to the next step.
1. Check whether reports containing vulnerability severites based on the value of parameter `severities` or not. If VEX documents exist, check whether there are vulnerabilities can be filtered out or be counted in.
- If yes, whether the deployment of the image is denied or not depends on enforcement actions.
- If no, verification succeeds with a successful log.

#### Error messages

| Scenarios | Error messages |
| --------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
| Vulnerability report is not found | Vulnerability reports are not found for the image <reference_by_digest> |
| Vulnerability report is not in SARIF format | The vulnerability report format is not supported. Please use the SARIF format. Image: <reference_by_digest> |
| Signature verification is enabled, but signatures are not found | Signatures are not found or signature types are not supported for the image <reference_by_digest> |
| Signature verification failure | Failed to validate signature, signature manifest <Reference>, reason: <Reason> |
| The vulnerability report is older than maxReportAge | The vulnerability report is older than <maxReportAge>, image: <reference_by_digest> |
| Found CVE in the CVE deny list | The image contains CVEs that are not allowed, image: <reference_by_digest> |
| The vulnerabilities do not meet the severity criteria | The image <reference_by_digest> contains vulnerabilities that do not meet the severity criteria. <severties> are not allowed. |

## Requirements

### Functional Requirements

| Feature | Priority | Dependency |
| ------------------------------------------------------------------ | -------- | ---------- |
| New parameterized Rego policy for verifying vulnerability report | P0 | |
| Pull the latest vulnerability report | P0 | Requiring adding creation timestamp annotation when storing reports in Registries |
| Support vulnerability report in SARIF format | P0 | |
| Support chaging the value of policy parameters in real-time | P0 | |
| Support VEX documents in openVex format | P1 | |
| Support VEX documents in cyclonedx VEX format | P1 | |

### Test Requirements

#### Performance testing

Assumptions:
- Vulnerability report is generated every 24 hours for an image
- Every vulnerability report is signed with 1 signature
- Each image has 1 signature
- A new version of image is published every 30 days
- One Pod has 10 images

Based on the assumptions, after image is stored in a registry for consumption and before a new version is published, it could have 30 vuln reports + 30 vuln report signatures + 1 image signature = 61 referrers

Variables:
- The Size of a single vulnerability report: 1M, 2M, 3M (need suggestions on rational sizes)
- Ratify instance: single instance, HA

Targets:
- What is maximum number of pods supported (with acceptable request duration) for a single Ratify instance, and what is the corresponding request duration in P95, CPU resource utiliaztion, MEM resource utilization for 200 Pods, 2000 pods and up to the limit. Test different size of vulnerability reports.
- What is maximum number of pods supported (with acceptable request duration) for Ratify HA with 3 replicas, and what is the request duration in P95, CPU resource utiliaztion, MEM resource utilization for 200 Pods, 2000 pods and up to the limit. Test different size of vulnerability reports.