HackMD- wes hayutin·Last edited by wes hayutin on Jun 7, 2023Linked with GitHubContributed by
Log in to edit or delete your comments and be notified of replies.
There is no commentSelect some text and then click Comment, or simply add a comment to this page from below to start a discussion.storage class OADP-1821 notes #2
context
The dpa volumeOptionsForStorageClasses is for volsync operations only.
The volumes are originally create w/ gp2-csi-copy-1 in the application namespace.
When and ONLY when volumes are used in the protected namespace ( openshift-adp ) volsync will use the storageclass gp2-csi-copy-2
volumeOptionsForStorageClasses parameters are volsync ONLY parameters and used in the context of volsync only.
volumeOptionsForStorageClasses:
gp2-csi-copy-1:
destinationVolumeOptions:
storageClassName: gp2-csi-copy-2
sourceVolumeOptions:
storageClassName: gp2-csi-copy-2
[whayutin@thinkdoe minimal-8csivol]$ oc get sc
NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
gp2-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 13d
gp2-csi-copy-1 ebs.csi.aws.com Delete WaitForFirstConsumer true 4h44m
gp2-csi-copy-2 ebs.csi.aws.com Delete WaitForFirstConsumer true 4h43m
gp3-csi ebs.csi.aws.com Delete WaitForFirstConsumer true 13d
[whayutin@thinkdoe minimal-8csivol]$ oc get dpa -n openshift-adp dpa-sample -o yaml
apiVersion: oadp.openshift.io/v1alpha1
kind: DataProtectionApplication
metadata:
creationTimestamp: "2023-06-05T17:22:53Z"
generation: 8
name: dpa-sample
namespace: openshift-adp
resourceVersion: "15356595"
uid: 0acbe7e3-6d14-429e-975d-786187f6b905
spec:
backupLocations:
- velero:
config:
profile: default
region: us-east-1
credential:
key: cloud
name: cloud-credentials
default: true
objectStorage:
bucket: cvpbucket
prefix: velero
provider: aws
configuration:
restic:
enable: false
velero:
defaultPlugins:
- openshift
- aws
- csi
- vsm
featureFlags:
- EnableCSI
features:
dataMover:
credentialName: restic-secret
enable: true
volumeOptionsForStorageClasses:
gp2-csi-copy-1:
destinationVolumeOptions:
storageClassName: gp2-csi-copy-2
sourceVolumeOptions:
storageClassName: gp2-csi-copy-2
snapshotLocations:
- velero:
config:
profile: default
region: us-east-1
provider: aws
status:
conditions:
- lastTransitionTime: "2023-06-05T17:22:54Z"
message: Reconcile complete
reason: Complete
status: "True"
type: Reconciled
setup
screen capture of test: https://file.rdu.redhat.com/~whayutin/OADP/OADP-1821.webm
window #1: watch -n 5 oc get pvc -n minimal-8csivol
window #2: watch -n 5 oc get pvc -n openshift-adp
window #3: watch -n 5 datamover_resources.sh -[b,r] -d
window #4: cli
machine state
clean
[whayutin@thinkdoe minimal-8csivol]$ oc get backup -n openshift-adp
No resources found in openshift-adp namespace.
[whayutin@thinkdoe minimal-8csivol]$ oc get restore -n openshift-adp
No resources found in openshift-adp namespace.
[whayutin@thinkdoe minimal-8csivol]$ s3cmd ls s3://cvpbucket/
[whayutin@thinkdoe minimal-8csivol]$
[whayutin@thinkdoe minimal-8csivol]$ oc delete vsb -A --all; oc delete vsr -A --all; oc delete vsc -A --all; oc delete vs -A --all; oc delete replicationsources.volsync.backube -A --all; oc delete replicationdestination.volsync.backube -A --all
No resources found
No resources found
No resources found
No resources found
No resources found
No resources found
[whayutin@thinkdoe minimal-8csivol]$
hen deleting "minimal-8csivol.yaml": namespaces "minimal-8csivol" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": deployments.apps "minimal-8csivol" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume1" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume2" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume3" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume4" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume5" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume6" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume7" not found
Error from server (NotFound): error when deleting "minimal-8csivol.yaml": persistentvolumeclaims "volume8" not found
create minimal-8csivol.yaml
[whayutin@thinkdoe minimal-8csivol]$ oc create -f minimal-8csivol.yaml
namespace/minimal-8csivol created
deployment.apps/minimal-8csivol created
persistentvolumeclaim/volume1 created
persistentvolumeclaim/volume2 created
persistentvolumeclaim/volume3 created
persistentvolumeclaim/volume4 created
persistentvolumeclaim/volume5 created
persistentvolumeclaim/volume6 created
persistentvolumeclaim/volume7 created
persistentvolumeclaim/volume8 created
[whayutin@thinkdoe minimal-8csivol]$ oc get pvc -n minimal-8csivol
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
volume1 Bound pvc-2af3bb5b-d3d9-4b05-8038-c8a5351e0fd8 1Gi RWO gp2-csi-copy-1 14s
volume2 Bound pvc-5c622a35-f48e-45c1-9be6-ebcae40629a2 1Gi RWO gp2-csi-copy-1 14s
volume3 Bound pvc-6a4e9a8a-a73e-41ce-944a-71c63b16b2bc 1Gi RWO gp2-csi-copy-1 14s
volume4 Bound pvc-a11c6e4d-1e82-4e81-9fd5-84c5ba62c1a7 1Gi RWO gp2-csi-copy-1 14s
volume5 Bound pvc-9e01f16e-ceee-47cc-945f-a764be6cd0a2 1Gi RWO gp2-csi-copy-1 14s
volume6 Bound pvc-9039b911-3ab1-40e1-bd41-062e5956ecd5 1Gi RWO gp2-csi-copy-1 14s
volume7 Bound pvc-264d2018-cabf-46ab-be21-28a432096cc4 1Gi RWO gp2-csi-copy-1 14s
volume8 Bound pvc-b1672d20-9185-43c7-af96-865e4c24a921 1Gi RWO gp2-csi-copy-1 14s
[whayutin@thinkdoe minimal-8csivol]$
[whayutin@thinkdoe minimal-8csivol]$
[whayutin@thinkdoe minimal-8csivol]$ ls
minimal-3csivol-backup.yaml minimal-3csivol.yaml minimal-8csivol-restore.yaml
minimal-3csivol-restore.yaml minimal-8csivol-backup.yaml minimal-8csivol.yaml
[whayutin@thinkdoe minimal-8csivol]$ oc create -f minimal-8csivol-backup.yaml
backup.velero.io/minimal-8csivol-1 created
[whayutin@thinkdoe minimal-8csivol]$ oc get backup.velero.io/minimal-8csivol-1 -n openshift-adp -o yaml
apiVersion: velero.io/v1
kind: Backup
metadata:
annotations:
velero.io/source-cluster-k8s-gitversion: v1.25.8+27e744f
velero.io/source-cluster-k8s-major-version: "1"
velero.io/source-cluster-k8s-minor-version: "25"
creationTimestamp: "2023-06-07T18:26:19Z"
generation: 4
labels:
velero.io/storage-location: dpa-sample-1
name: minimal-8csivol-1
namespace: openshift-adp
resourceVersion: "15450289"
uid: 13e3e32a-c86b-49c3-9c8f-2415981ebf88
spec:
csiSnapshotTimeout: 10m0s
defaultVolumesToFsBackup: false
includedNamespaces:
- minimal-8csivol
itemOperationTimeout: 1h0m0s
storageLocation: dpa-sample-1
ttl: 720h0m0s
volumeSnapshotLocations:
- dpa-sample-1
status:
backupItemOperationsAttempted: 8
backupItemOperationsCompleted: 8
csiVolumeSnapshotsAttempted: 8
csiVolumeSnapshotsCompleted: 8
expiration: "2023-07-07T18:26:19Z"
formatVersion: 1.1.0
phase: Finalizing
startTimestamp: "2023-06-07T18:26:19Z"
version: 1
Every 5.0s: oc get pvc -n openshift-adp thinkdoe: Wed Jun 7 12:35:21 2023
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
volsync-vsb-7ngck-rep-src-cache Bound pvc-4fdd9ef2-fd3b-4711-a80c-7f7bc18c8da3 1Gi RWO gp2-csi-copy-2 2m38s
volsync-vsb-c4rbr-rep-src-cache Bound pvc-78042f80-1246-4bea-8f66-97f5ec5428ee 1Gi RWO gp2-csi-copy-2 5m23s
volsync-vsb-d5x2d-rep-src-cache Bound pvc-077b26b3-356a-467b-9475-220e98bed220 1Gi RWO gp2-csi-copy-2 7m30s
volsync-vsb-gqlqf-rep-src-cache Bound pvc-73670130-ff69-4414-bcb9-60a656971763 1Gi RWO gp2-csi-copy-2 91s
volsync-vsb-hk4sn-rep-src-cache Bound pvc-b47ee505-bae5-4364-bd10-a7fcb061552e 1Gi RWO gp2-csi-copy-2 4m1s
volsync-vsb-l8nzt-rep-src-cache Bound pvc-7d1092c8-426f-4ff4-8644-560463b31531 1Gi RWO gp2-csi-copy-2 2m12s
volsync-vsb-swzlq-rep-src-cache Bound pvc-75e29343-54a9-4db5-8ca2-00e966b2edc9 1Gi RWO gp2-csi-copy-2 4m17s
volsync-vsb-vp588-rep-src-cache Bound pvc-96f6fe75-8f2e-44cc-a866-6f89ec17ccc5 1Gi RWO gp2-csi-copy-2 3m
Every 5.0s: oc get pvc -n minimal-8csivol thinkdoe: Wed Jun 7 12:35:35 2023
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
volume1 Bound pvc-2af3bb5b-d3d9-4b05-8038-c8a5351e0fd8 1Gi RWO gp2-csi-copy-1 10m
volume2 Bound pvc-5c622a35-f48e-45c1-9be6-ebcae40629a2 1Gi RWO gp2-csi-copy-1 10m
volume3 Bound pvc-6a4e9a8a-a73e-41ce-944a-71c63b16b2bc 1Gi RWO gp2-csi-copy-1 10m
volume4 Bound pvc-a11c6e4d-1e82-4e81-9fd5-84c5ba62c1a7 1Gi RWO gp2-csi-copy-1 10m
volume5 Bound pvc-9e01f16e-ceee-47cc-945f-a764be6cd0a2 1Gi RWO gp2-csi-copy-1 10m
volume6 Bound pvc-9039b911-3ab1-40e1-bd41-062e5956ecd5 1Gi RWO gp2-csi-copy-1 10m
volume7 Bound pvc-264d2018-cabf-46ab-be21-28a432096cc4 1Gi RWO gp2-csi-copy-1 10m
volume8 Bound pvc-b1672d20-9185-43c7-af96-865e4c24a921 1Gi RWO gp2-csi-copy-1 10m
delete app and restore
[whayutin@thinkdoe minimal-8csivol]$ oc delete -f minimal-8csivol.yaml
namespace "minimal-8csivol" deleted
deployment.apps "minimal-8csivol" deleted
persistentvolumeclaim "volume1" deleted
persistentvolumeclaim "volume2" deleted
persistentvolumeclaim "volume3" deleted
persistentvolumeclaim "volume4" deleted
persistentvolumeclaim "volume5" deleted
persistentvolumeclaim "volume6" deleted
persistentvolumeclaim "volume7" deleted
persistentvolumeclaim "volume8" deleted
[whayutin@thinkdoe minimal-8csivol]$ oc delete vsb -A --all; oc delete vsr -A --all; oc delete vsc -A --all; oc delete vs -A --all; oc delete replicationsources.volsync.backube -A --all; oc delete replicationdestination.volsync.backube -A --all
No resources found
No resources found
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-33ddc8f6-1ddf-4b53-b189-62c4428b83df" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-383c5dfa-3dfe-497b-aabd-e8566178bc48" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-4c531dfc-f8d2-41c7-8dce-d61c590ba6e3" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-575eb81d-fc71-47c0-a120-076dd10acf90" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-9095e586-ac44-491b-8cfd-b86cd2016f47" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-9e4f1bac-d88d-453f-b09b-712528463da9" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-b7ac405c-8632-453a-9cb5-3037d889e995" deleted
volumesnapshotcontent.snapshot.storage.k8s.io "snapcontent-f0a1bce2-5549-4c03-9935-70efd26bc581" deleted
No resources found
replicationsource.volsync.backube "vsb-7ngck-rep-src" deleted
replicationsource.volsync.backube "vsb-c4rbr-rep-src" deleted
replicationsource.volsync.backube "vsb-d5x2d-rep-src" deleted
replicationsource.volsync.backube "vsb-gqlqf-rep-src" deleted
replicationsource.volsync.backube "vsb-hk4sn-rep-src" deleted
replicationsource.volsync.backube "vsb-l8nzt-rep-src" deleted
replicationsource.volsync.backube "vsb-swzlq-rep-src" deleted
replicationsource.volsync.backube "vsb-vp588-rep-src" deleted
No resources found
[whayutin@thinkdoe minimal-8csivol]$
[whayutin@thinkdoe minimal-8csivol]$
[whayutin@thinkdoe minimal-8csivol]$ oc create -f minimal-8csivol-restore.yaml
restore.velero.io/minimal-8csivol-restore-1 created
[whayutin@thinkdoe minimal-8csivol]$ oc get restore.velero.io/minimal-8csivol-restore-1 -n openshift-adp -o yaml
apiVersion: velero.io/v1
kind: Restore
metadata:
creationTimestamp: "2023-06-07T18:37:13Z"
generation: 4
name: minimal-8csivol-restore-1
namespace: openshift-adp
resourceVersion: "15452597"
uid: b371309e-36d7-42f5-88f1-95ae6283311b
spec:
backupName: minimal-8csivol-1
excludedResources:
- nodes
- events
- events.events.k8s.io
- backups.velero.io
- restores.velero.io
- resticrepositories.velero.io
- csinodes.storage.k8s.io
- volumeattachments.storage.k8s.io
- backuprepositories.velero.io
itemOperationTimeout: 1h0m0s
restorePVs: true
status:
phase: InProgress
progress:
itemsRestored: 1
totalItems: 76
startTimestamp: "2023-06-07T18:37:13Z"
[whayutin@thinkdoe minimal-8csivol]$
final state
Every 5.0s: oc get pvc -n openshift-adp thinkdoe: Wed Jun 7 12:40:18 2023
No resources found in openshift-adp namespace.
Every 5.0s: oc get pvc -n minimal-8csivol thinkdoe: Wed Jun 7 12:40:33 2023
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE
volume1 Bound pvc-4c84a276-c727-493a-bc6f-35d0f9a2c2af 1Gi RWO gp2-csi-copy-1 41s
volume2 Bound pvc-02bcc29a-c563-46ff-b587-4b06df0f45c1 1Gi RWO gp2-csi-copy-1 41s
volume3 Bound pvc-848ebeef-4d29-417d-9506-b3c2875101b2 1Gi RWO gp2-csi-copy-1 41s
volume4 Bound pvc-ef2c0ecc-55fc-4d16-ab2b-054ee17ff8c3 1Gi RWO gp2-csi-copy-1 40s
volume5 Bound pvc-27d5439f-dd07-4aa3-9af5-8e30e6a3360b 1Gi RWO gp2-csi-copy-1 40s
volume6 Bound pvc-9754ac65-bc2c-4e4f-8e73-4f5174cbdc3a 1Gi RWO gp2-csi-copy-1 40s
volume7 Bound pvc-7178e6fa-fe20-4e2f-9ecb-7b182291efb3 1Gi RWO gp2-csi-copy-1 40s
volume8 Bound pvc-519e7169-8c58-4486-bed7-eb9c26104b52 1Gi RWO gp2-csi-copy-1 39s
Every 5.0s: datamover_resources.sh -r -d thinkdoe: Wed Jun 7 12:40:38 2023
Get Restores:
NAME BACKUP STATUS STARTED COMPLETED
ERRORS WARNINGS CREATED SELECTOR
minimal-8csivol-restore-1 minimal-8csivol-1 Completed 2023-06-07 18:37:13 +0000 UTC 2023-06-07 18:40:27 +0000 U
TC 0 14 2023-06-07 18:37:13 +0000 UTC <none>
No resources found
Total VSR: 0
No resources found
Completed: 0
No resources found
InProgress: 0
No resources found
SnapshotRestoreDone: 0
VSR STATUS
Completed: 0
Processing: 0
Queued: 0
No resources found
Total ReplicationDestinations: 0
***** VOLUME SNAPSHOT CONTENTS ******
[whayutin@thinkdoe minimal-8csivol]$ oc get restore.velero.io/minimal-8csivol-restore-1 -n openshift-adp -o yaml
apiVersion: velero.io/v1
kind: Restore
metadata:
creationTimestamp: "2023-06-07T18:37:13Z"
generation: 24
name: minimal-8csivol-restore-1
namespace: openshift-adp
resourceVersion: "15456229"
uid: b371309e-36d7-42f5-88f1-95ae6283311b
spec:
backupName: minimal-8csivol-1
excludedResources:
- nodes
- events
- events.events.k8s.io
- backups.velero.io
- restores.velero.io
- resticrepositories.velero.io
- csinodes.storage.k8s.io
- volumeattachments.storage.k8s.io
- backuprepositories.velero.io
itemOperationTimeout: 1h0m0s
restorePVs: true
status:
completionTimestamp: "2023-06-07T18:40:27Z"
phase: Completed
progress:
itemsRestored: 77
totalItems: 77
restoreItemOperationsAttempted: 8
restoreItemOperationsCompleted: 8
startTimestamp: "2023-06-07T18:37:13Z"
warnings: 14
[whayutin@thinkdoe minimal-8csivol]$ oc get all -n minimal-8csivol
NAME READY STATUS RESTARTS AGE
pod/minimal-8csivol-6d848c77f8-4jvt7 1/1 Running 0 116s
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/minimal-8csivol 1/1 1 1 116s
NAME DESIRED CURRENT READY AGE
replicaset.apps/minimal-8csivol-6d848c77f8 1 1 1 116s
[whayutin@thinkdoe minimal-8csivol]$
