Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Use storage class from annotation for host-assisted PVC operation #3585

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: Use storage class from annotation for host-assisted PVC operation #3585

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions pkg/controller/common/util.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,9 @@ const (

// AnnSourceVolumeMode is the volume mode of the source PVC specified as an annotation on snapshots
AnnSourceVolumeMode = AnnAPIGroup + "/storage.import.sourceVolumeMode"
// AnnSnapshotRestoreStorageClass indicates the storage class to use for restoring snapshots.
// This annotation should be added to DataVolumes for the temporary host-assisted source PVC.
AnnSnapshotRestoreStorageClass = AnnAPIGroup + "/snapshot.restore.storageClass"

// AnnOpenShiftImageLookup is the annotation for OpenShift image stream lookup
AnnOpenShiftImageLookup = "alpha.image.policy.openshift.io/resolve-names"
Expand Down
9 changes: 7 additions & 2 deletions pkg/controller/datavolume/snapshot-clone-controller.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -418,7 +418,7 @@ func (r *SnapshotCloneReconciler) createTempHostAssistedSourcePvc(dv *cdiv1.Data
if err := r.client.Get(context.TODO(), types.NamespacedName{Name: *snapshot.Spec.VolumeSnapshotClassName}, vsc); err != nil {
return err
}
sc, err := r.getStorageClassCorrespondingToSnapClass(vsc.Driver)
sc, err := r.getStorageClassCorrespondingToSnapClass(dv, vsc.Driver)
if err != nil {
return err
}
Expand Down Expand Up @@ -450,7 +450,12 @@ func (r *SnapshotCloneReconciler) createTempHostAssistedSourcePvc(dv *cdiv1.Data
return nil
}

func (r *SnapshotCloneReconciler) getStorageClassCorrespondingToSnapClass(driver string) (string, error) {
func (r *SnapshotCloneReconciler) getStorageClassCorrespondingToSnapClass(dv *cdiv1.DataVolume, driver string) (string, error) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this only covers the non-CSI path (initLegacyClone), I assume you're more interested in the CSI one?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The changes I made are for cases where TempHostAssistedSourcePvc is created, i.e., the csi driver is not present, or the csi driver for the source and target is mismatched. IIRC both are triggered by initLegacyClone . the first one is my use case

happy to work on any similar cases. Are you referring to the remaining case, which is the csi driver for target pvc and snapshot match? It tries for a direct snapshot to target pvc restore , but it fails because of a csi incompatibility issue ?

Copy link
Collaborator

@akalenyu akalenyu Jan 8, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh right, I forgot the issue details. Let me stress again that what you're describing is very unusual today

the csi driver is not present

It tries for a direct snapshot to target pvc restore , but it fails because of a csi incompatibility issue ?

Yes, here's a ref to this path

containerized-data-importer/pkg/controller/clone/planner.go

Line 835 in dcf7cfb

func getStorageClassNameForTempSourceClaim(ctx context.Context, vsc *snapshotv1.VolumeSnapshotContent, client client.Client) (string, error) {

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I will work on this

Copy link
Collaborator

@akalenyu akalenyu Jan 9, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So just to poke at the setup a little bit again, the provisioner has VolumeSnapshots support (which I thought implies it's CSI) but there's no CSIDriver object?

Copy link
Author

@TheRealSibasishBehera TheRealSibasishBehera Jan 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's the case. The provisioner is openebs mayastor The deployment manifest does not contain a CSIDriver packaged within

That sounds unintended, how would you then configure things like fsGroupPolicy? https://kubernetes-csi.github.io/docs/support-fsgroup.html

These are filled in as parameters for the storage class (not exactly the same names but similar functionality that made it possible before GA, I believe). I have confirmed with their community that they do not have a CSIDriver.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt think about also using pv.spec.CSI since it is directly populated in when using a csi driver

I don't think it's possible since a PV does not exist at the time CDI is interested in this

so its not evaluated on fly for each dv/pvc ?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the motivation to avoid a CSIDriver object.. AFAIK it's a requirement https://kubernetes-csi.github.io/docs/csi-driver-object.html

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

wdyt think about also using pv.spec.CSI since it is directly populated in when using a csi driver

I don't think it's possible since a PV does not exist at the time CDI is interested in this

so its not evaluated on fly for each dv/pvc ?

It's evaluated prior to PVC creation to understand if we can use populators

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand the motivation to avoid a CSIDriver object.. AFAIK it's a requirement https://kubernetes-csi.github.io/docs/csi-driver-object.html

Yeah I agree it’s a standard and should be supported 😞
This seems like a case of delayed adoption to me, as it used to work without it

if sc, found := dv.Annotations[cc.AnnSnapshotRestoreStorageClass]; found {
r.log.V(3).Info("Using storage class from annotation for host-assisted source PVC operation", "storageClass.Name", sc)
return sc, nil
}

matches := []storagev1.StorageClass{}

storageClasses := &storagev1.StorageClassList{}
Expand Down
41 changes: 41 additions & 0 deletions pkg/controller/datavolume/snapshot-clone-controller_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,6 +168,47 @@ var _ = Describe("All DataVolume Tests", func() {
Expect(err).ToNot(HaveOccurred())
})

It("Should use the storage class used in annotation AnnSnapshotRestoreStorageClass if present", func() {
dv := newCloneFromSnapshotDataVolume("test-dv")
scName := "testsc"
expectedSnapshotClass := "snap-class"
sc := CreateStorageClassWithProvisioner(scName, map[string]string{
AnnDefaultStorageClass: "true",
}, map[string]string{}, "csi-plugin")
targetScName := "targetsc"
scSameProvisioner := sc.DeepCopy()
scSameProvisioner.Name = "same-provisioner-as-source-sc"
scUsedInAnnotation := sc.DeepCopy()
scUsedInAnnotation.Name = "sc-used-in-annotation"
dv.Annotations[AnnSnapshotRestoreStorageClass] = scUsedInAnnotation.Name

tsc := CreateStorageClassWithProvisioner(targetScName, map[string]string{}, map[string]string{}, "another-csi-plugin")
sp := createStorageProfile(scName, []corev1.PersistentVolumeAccessMode{corev1.ReadOnlyMany}, BlockMode)
sp2 := createStorageProfile(targetScName, []corev1.PersistentVolumeAccessMode{corev1.ReadOnlyMany}, BlockMode)
sp3 := createStorageProfile(scSameProvisioner.Name, []corev1.PersistentVolumeAccessMode{corev1.ReadOnlyMany}, BlockMode)
sp4 := createStorageProfile(scUsedInAnnotation.Name, []corev1.PersistentVolumeAccessMode{corev1.ReadOnlyMany}, BlockMode)
dv.Spec.PVC.StorageClassName = &targetScName
snapshot := createSnapshotInVolumeSnapshotClass("test-snap", metav1.NamespaceDefault, &expectedSnapshotClass, nil, nil, true)
snapClass := createSnapshotClass(expectedSnapshotClass, nil, "csi-plugin")
reconciler = createSnapshotCloneReconciler(sc, scSameProvisioner, tsc, sp, sp2, sp3, sp4, dv, snapshot, snapClass, createDefaultVolumeSnapshotContent(), createVolumeSnapshotContentCrd(), createVolumeSnapshotClassCrd(), createVolumeSnapshotCrd())
_, err := reconciler.Reconcile(context.TODO(), reconcile.Request{NamespacedName: types.NamespacedName{Name: "test-dv", Namespace: metav1.NamespaceDefault}})
Expect(err).ToNot(HaveOccurred())
By("Verifying that temp host assisted source PVC is being created")
pvc := &corev1.PersistentVolumeClaim{}
tempPvcName := getTempHostAssistedSourcePvcName(dv)
err = reconciler.client.Get(context.TODO(), types.NamespacedName{Namespace: snapshot.Namespace, Name: tempPvcName}, pvc)
Expect(err).ToNot(HaveOccurred())
Expect(pvc.Labels[common.CDIComponentLabel]).To(Equal("cdi-clone-from-snapshot-source-host-assisted-fallback-pvc"))
Expect(pvc.Labels[common.AppKubernetesPartOfLabel]).To(Equal("testing"))
Expect(*pvc.Spec.StorageClassName).To(Equal(scUsedInAnnotation.Name))
By("Verifying that target host assisted PVC is being created")
err = reconciler.client.Get(context.TODO(), types.NamespacedName{Namespace: dv.Namespace, Name: dv.Name}, pvc)
Expect(err).ToNot(HaveOccurred())
Expect(pvc.Labels[common.AppKubernetesPartOfLabel]).To(Equal("testing"))
Expect(pvc.Annotations[AnnCloneRequest]).To(Equal(fmt.Sprintf("%s/%s", snapshot.Namespace, tempPvcName)))
Expect(*pvc.Spec.StorageClassName).To(Equal(targetScName))
})

It("Should clean up host assisted source temp PVC when done", func() {
dv := newCloneFromSnapshotDataVolume("test-dv")
dv.Status.Phase = cdiv1.Succeeded
Expand Down