# OADP SSE-C * server side encryption, customer provided key ## AWS Default Encryption on S3 * https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-encryption.html ## links * https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerSideEncryptionCustomerKeys.html#sse-c-highlights * https://docs.aws.amazon.com/AmazonS3/latest/userguide/ServerSideEncryptionCustomerKeys.html * cookbook * SSE-C https://catalog.us-east-1.prod.workshops.aws/workshops/aad9ff1e-b607-45bc-893f-121ea5224f24/en-US/s3/serverside/ssec * Client-side encryption https://catalog.us-east-1.prod.workshops.aws/workshops/aad9ff1e-b607-45bc-893f-121ea5224f24/en-US/s3/clientside * https://github.com/vmware-tanzu/velero-plugin-for-aws/blob/main/backupstoragelocation.md ``` # Specify the file that contains the SSE-C customer key to enable customer key encryption of the backups # stored in S3. The referenced file should contain a 32-byte string. # # The customerKeyEncryptionFile points to a mounted secret within the velero container. # Add the below values to the velero cloud-credentials secret: # customer-key: <your_b64_encoded_32byte_string> # The default value below points to the already mounted secret. # # Cannot be used in conjunction with kmsKeyId. # # Optional (defaults to "", which means SSE-C is disabled). customerKeyEncryptionFile: "/credentials/customer-key" ``` * WHY use customer-key? By default aws s3 is encrypted ( look at the properties ). We need a level of encryption that is not unlocked w/ the users aws key/secret-key. * Create the Key: ``` dd if=/dev/urandom bs=1 count=32 > sse.key cat sse.key | base64 > sse_encoded.key ln -s sse_encoded.key customer-key ``` * Create the cloud-credentials secret. ``` oc create secret generic cloud-credentials --namespace openshift-adp --from-file cloud=<path>/openshift_aws_credentials,customer-key=<path>/sse_encoded.key ``` * Alternatively, update existing cloud-credentials secret data. `customer-key: <value of customer-key>` ``` whayutin@thinkdoe:~$ oc get secrets -n openshift-adp cloud-credentials -o yaml apiVersion: v1 data: cloud: W2Rfa2V5X2lkPSJBS0lBVkJRWUIyRkQ0TlFHRFFPQiIKYXdzX3NlY3JldF9hY2Nlc3Nfa2V5P<snip>rUE1mNWVSbTN5K2FpeWhUTUQyQk1WZHBOIgo= customer-key: v+<snip>TFIiq6aaXPbj8dhos= kind: Secret metadata: creationTimestamp: "2024-04-16T14:19:44Z" labels: openshift-adp.dataprotectionapplication: dpa-sample openshift.io/oadp: "True" name: cloud-credentials namespace: openshift-adp resourceVersion: "26734766" uid: 042cc208-b71f-4483-9107-0d8dded00d36 type: Opaque whayutin@thinkdoe:~$ ``` ``` cat customer-key v+YZbdgZBw99e01KvYYDKIgItDTFIiq6aaXPbj8dhos= ``` * DPA example ``` spec: backupLocations: - velero: config: customerKeyEncryptionFile: /credentials/customer-key profile: default ``` WOOT * Backup and Restore works ## testing * Create a test file ``` echo "encrypt me please" > test.txt ``` * upload the test file ``` aws s3api put-object \ --bucket cvpbucketuswest2 \ --key test.txt \ --body test.txt \ --sse-customer-key fileb://sse.key \ --sse-customer-algorithm AES256 ``` * Try to download the file * via the s3 web console * `s3cmd get s3://cvpbucketuswest2/test.txt test.txt` This should fail as the file is now encrypted w/ an extra key * Download the file with key ``` aws s3api get-object \ --bucket cvpbucketuswest2 \ --key test.txt \ --sse-customer-key fileb://sse.key \ --sse-customer-algorithm AES256 \ downloaded.txt ``` ``` cat downloaded.txt encrypt me please ``` * The same download call can be executed on velero backed up files ``` aws s3api get-object \ --bucket cvpbucketuswest2 \ --key velero/backups/mysql-persistent-customerkeyencryptionfile4/mysql-persistent-customerkeyencryptionfile4.tar.gz \ --sse-customer-key fileb://sse.key \ --sse-customer-algorithm AES256 \ --debug \ velero_download.tar.gz ```` ```` tar -zxvf velero_download.tar.gz ```` ``` ls metadata resources velero_download.tar.gz whayutin@thinkdoe:/tmp/wes$ less resources/ clusterserviceversions.operators.coreos.com/ persistentvolumes/ configmaps/ pods/ customresourcedefinitions.apiextensions.k8s.io/ replicasets.apps/ datauploads.velero.io/ replicationcontrollers/ deploymentconfigs.apps.openshift.io/ rolebindings.authorization.openshift.io/ deployments.apps/ rolebindings.rbac.authorization.k8s.io/ endpoints/ routes.route.openshift.io/ endpointslices.discovery.k8s.io/ secrets/ events/ securitycontextconstraints.security.openshift.io/ namespaces/ serviceaccounts/ persistentvolumeclaims/ services/ ``` ## GOTCHA's * If the sse.key is changed the velero server will need to be restarted to mount the updated key. ## Additional references https://www.bleepingcomputer.com/news/security/ransomware-abuses-amazon-aws-feature-to-encrypt-s3-buckets/ https://arcticwolf.com/resources/blog/ransomware-campaign-encrypting-amazon-s3-buckets-using-sse-c/ https://aws.amazon.com/blogs/storage/protecting-data-with-amazon-s3-object-lock/ ## aws s3 object locking * https://aws.amazon.com/blogs/storage/protecting-data-with-amazon-s3-object-lock/ * https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock.html?icmpid=docs_amazons3_console * https://github.com/vmware-tanzu/velero/issues/7400 * https://github.com/vmware-tanzu/velero-plugin-for-aws/pull/194 *