--- title: Build, sign, and verify container images using Notary and Azure Key Vault description: This tutorial is based on https://learn.microsoft.com/en-us/azure/container-registry/container-registry-tutorial-sign-build-push with the update to using notation beta.1 release --- # Build, sign, and verify container images using Notary and Azure Key Vault (Preview) The Azure Key Vault (AKV) is used to store a signing key that can be utilized by **notation** with the notation AKV plugin (azure-kv) to sign and verify container images and other artifacts. The Azure Container Registry (ACR) allows you to attach these signatures using the **az** or **oras** CLI commands. The signed containers enable users to assure deployments are built from a trusted entity and verify artifact hasn't been tampered with since their creation. The signed artifact ensures integrity and authenticity before the user pulls an artifact into any environment and avoid attacks. In this tutorial: > [!div class="checklist"] > * Store a signing certificate in Azure Key Vault > * Sign a container image with notation > * Verify a container image signature with notation ## Prerequisites > * Install, create and sign in to [ORAS artifact enabled registry](./container-registry-oras-artifacts.md#create-oras-artifact-enabled-registry) > * Create or use an [Azure Key Vault](../key-vault/general/quick-create-cli.md) >* This tutorial can be run in the [Azure Cloud Shell](https://portal.azure.com/#cloudshell/) ## The workflow of this tutorial ```mermaid sequenceDiagram participant AKV participant Alice participant AKV_Plugin participant Registry note over AKV, Registry: Install notation and AKV plugin Alice->>Alice: Install notation/AKV plugin, and configure ENV note over AKV, Registry: Store or create a certificate in AKV for signing and verifying Alice->>AKV: Create a self-signed certificate Alice->>AKV: Retrieve the key id for the certificate Alice->>AKV: Download the certificate for verification note over AKV, Registry: Build and Push an container image Alice->>Alice: Build a container image Alice->>Registry: Push the container image note over AKV, Registry: Sign an container image and store the signature Alice->>Alice: Add a signing key referecing key id Alice->>AKV_Plugin: Sign the container image AKV_Plugin->>Alice: Return the raw signature Alice->>Alice: Pack the signature with an envelope Alice->>Registry: Store the signature associated with the container image note over AKV, Registry: Verify conainer image against signatures Alice->>Alice: Configure trust store Alice->>Alice: Configure trust policy Alice->>Registry: Verify the container image against signatures ``` ## Install the notation CLI and AKV plugin 1. Install notation 0.12.0-beta.1 with plugin support on a Linux environment. You can also download the package for other environments from the [release page](https://github.com/notaryproject/notation/releases/tag/v0.12.0-beta.1). ```bash # Download, extract and install curl -Lo notation.tar.gz https://github.com/notaryproject/notation/releases/download/v0.12.0-beta.1/notation_0.12.0-beta.1_linux_amd64.tar.gz tar xvzf notation.tar.gz # Copy the notation cli to the desired bin directory in your PATH cp ./notation /usr/local/bin ``` 2. Install the notation Azure Key Vault plugin for remote signing and verification. > [!NOTE] > The plugin directory varies depending upon the operating system being used. The directory path below assumes Ubuntu. > Please read the [notation config article](https://github.com/notaryproject/notation/blob/main/specs/notation-config.md) for more information. ```bash # Create a directory for the plugin mkdir -p ~/.config/notation/plugins/azure-kv # Download the plugin curl -Lo notation-azure-kv.tar.gz \ https://github.com/Azure/notation-azure-kv/releases/download/v0.4.0-beta.1/notation-azure-kv_0.4.0-beta.1_Linux_amd64.tar.gz # Extract to the plugin directory tar xvzf notation-azure-kv.tar.gz -C ~/.config/notation/plugins/azure-kv notation-azure-kv ``` 3. List the available plugins and verify that the plugin is available. ```bash notation plugin ls ``` ## Configure environment variables > [!NOTE] > For easy execution of commands in the tutorial, provide values for the Azure resources to match the existing ACR and AKV resources. 1. Configure AKV resource names. ```bash # Name of the existing Azure Key Vault used to store the signing keys AKV_NAME=<your-unique-keyvault-name> # New desired key name used to sign and verify KEY_NAME=wabbit-networks-io CERT_PATH=./${KEY_NAME}.pem ``` 2. Configure ACR and image resource names. ```bash # Name of the existing registry example: myregistry.azurecr.io ACR_NAME=<your-registry-name> # Existing full domain of the ACR REGISTRY=$ACR_NAME.azurecr.io # Container name inside ACR where image will be stored REPO=net-monitor TAG=v1 IMAGE=$REGISTRY/${REPO}:$TAG # Source code directory containing Dockerfile to build IMAGE_SOURCE=https://github.com/wabbit-networks/net-monitor.git#main ``` ## Store the signing certificate in AKV If you have an existing certificate, upload it to AKV. For more information on how to use your own signing key, see the [signing certificate requirements.](https://github.com/notaryproject/notaryproject/blob/main/signature-specification.md#certificate-requirements) Otherwise create an x509 self-signed certificate storing it in AKV for remote signing using the steps below. ### Create a self-signed certificate (Azure CLI) 1. Create a certificate policy file. Once the certificate policy file is executed as below, it creates a valid signing certificate compatible with **notation** in AKV. The EKU listed is for code-signing, but isn't required for notation to sign artifacts. ```bash cat <<EOF > ./my_policy.json { "issuerParameters": { "certificateTransparency": null, "name": "Self" }, "x509CertificateProperties": { "ekus": [ "1.3.6.1.5.5.7.3.3" ], "keyUsage": [ "digitalSignature" ], "subject": "CN=wabbit-networks.io,O=Notary,L=Seattle,ST=WA,C=US", "validityInMonths": 12 } } EOF ``` 1. Create the certificate. ```azure-cli az keyvault certificate create -n $KEY_NAME --vault-name $AKV_NAME -p @my_policy.json ``` 1. Get the Key ID for the certificate. ```bash KEY_ID=$(az keyvault certificate show -n $KEY_NAME --vault-name $AKV_NAME --query 'kid' -o tsv) ``` 4. Download public certificate. ```bash CERT_ID=$(az keyvault certificate show -n $KEY_NAME --vault-name $AKV_NAME --query 'id' -o tsv) az keyvault certificate download --file $CERT_PATH --id $CERT_ID --encoding PEM ``` ## Build and sign a container image 1. Build and push a new image with ACR Tasks. ```azure-cli az acr build -r $ACR_NAME -t $IMAGE $IMAGE_SOURCE ``` 2. Authenticate with your individual Azure AD identity to use an ACR token. ```azure-cli export USER_NAME="00000000-0000-0000-0000-000000000000" export PASSWORD=$(az acr login --name $ACR_NAME --expose-token --output tsv --query accessToken) notation login -u $USER_NAME -p $PASSWORD $REGISTRY ``` 3. Add a signing key referecing the Key ID ```bash notation key add $KEY_NAME --plugin azure-kv --id $KEY_ID ``` 4. List the keys to confirm. ```bash notation key ls ``` 5. Choose [COSE](https://datatracker.ietf.org/doc/html/rfc8152) signature format to sign the container image. - Sign the container image with the COSE signature envelope: ```bash notation sign --signature-format cose --key $KEY_NAME $IMAGE ``` ## View the signed images associated with signatures Signed images can be viewed with the `notation list` command ```bash notation list $IMAGE ``` Alternatively, signed images can be viewed with ORAS CLI ```bash oras login -u $USER_NAME -p $PASSWORD $REGISTRY oras discover -o tree $IMAGE ``` ## Verify the container image against signatures 1. Configure trust store ```bash STORE_TYPE="ca" STORE_NAME="wabbit-networks.io" notation certificate add --type $STORE_TYPE --store $STORE_NAME $CERT_PATH ``` 2. Configure trust policy ```bash cat <<EOF > $HOME/.config/notation/trustpolicy.json { "version": "1.0", "trustPolicies": [ { "name": "wabbit-networks-images", "registryScopes": [ "*" ], "signatureVerification": { "level" : "strict" }, "trustStores": [ "$STORE_TYPE:$STORE_NAME" ], "trustedIdentities": [ "*" ] } ] } EOF ``` 3. The notation command can also help to ensure the container image hasn't been tampered with since build time by comparing the `sha` with what is in the registry. ```bash notation verify $IMAGE ``` The sha256 result is a successful verification of the image using the trusted certificate. To get a verification failure, we'll remove the certificate utilized to sign the image. ```azure-cli notation cert delete --type $STORE_TYPE --store $STORE_NAME --all notation verify $IMAGE Error: signature verification failed for all the signatures associated with localhost:5000/net-monitor@sha256:3d403e0e2381245e44579568f470f4fe068fdd076bda7ca5a707c5a6fde86f0b ```