Creating Volumes from Snapshots should be (almost) instant

[cometship]

Correct me if I'm wrong, but shouldn't Volume creation be (almost) instant when they are created from Snapshots (with COW supported)?

Because now I can't use Velero to create PVC Backups (Velero creates PVC referencing Snapshot), as it copies the Snapshot image (created instantly) byte-for-byte over to a new image file, which just takes too long. I thought it would just do cp --reflink=always snap.img restored.img.

See:

rawfile-localpv/rawfile/utils/snapshot_manager.py

Lines 123 to 136 in 0593b89

	def restore_snapshot(
	self, volume_id: str, name: str, destination: Path, temporary: bool = False
	):
	"""Restore a snapshot"""
	chunk_size = 1024 * 1024
	snap_path = self._get_snapshot_path(volume_id, name, temporary)
	with open(snap_path, "rb") as src, open(destination, "wb") as dst:
	while True:
	buf = src.read(chunk_size)
	if not buf:
	break
	dst.write(buf)
	dst.flush()
	fsync(dst.fileno())

Used here:

rawfile-localpv/rawfile/utils/volume_manager.py

Lines 148 to 153 in 0593b89

	snapshot_manager.restore_snapshot(
	source_volume_id,
	snapshot_name,
	img_file,
	source_type == VolumeSource.volume,
	)

[tiagolobocastro]

Yeah I think we could do that if copy_on_write is enabled in the storage class and it's the same storage_pool as the source snapshot.

[cometship]

Could also use cp --reflink=auto. That would handle COW on/off and different/same storage pools?

[cometship]

Also because Velero aborted after 30min it created two tasks that got stuck printing "Error: Volume not ready" (or something similar) in the csi-driver container. Deleting tasks.json in the meta folder fixed the problem. I also manually needed to delete the folders containing the now unused PVC meta/images/snapshots.

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[cometship]

Not stale

[mhkarimi1383]

Hi

Multiple steps are here to ensure that if we want to remove a volume/snapshot It should be safe, so nobody requires the source after clone, and it should be a new independent volume

[cometship]

so nobody requires the source after clone

Could you explain what you mean with that? That Source can be deleted after Snapshot?

and it should be a new independent volume

But is this not the case with cp --reflink=auto|always? Source and Clone are independent.

[mhkarimi1383]

Could you explain what you mean with that? That Source can be deleted after Snapshot?

I mean Volume should not be depend to the data source, when removing snapshot/source volume, the volume that was based on that should not require that source to be existed

But is this not the case with cp --reflink=auto|always? Source and Clone are independent.

As I have understood, cp --reflink=always will make the destination to be depend on the source

[cometship]

I mean Volume should not be depend to the data source, when removing snapshot/source volume, the volume that was based on that should not require that source to be existed

Yes, obviously Snapshot and Source should be fully independent.

As I have understood, cp --reflink=always will make the destination to be depend on the source

They share the same data blocks, yes. But deleting/changing the source or the copy does not affect the other. The filesystem keeps track of shared data blocks and only frees them when no file references them anymore. Changing a file triggers CoW, modified parts are copied and at that moment these parts become independent.

[mhkarimi1383]

Sorry I was wrong about it, I will change it, When I have completed the API Server

Thanks