* Questions about how block devices use snapshots
@ 2023-01-09 12:45 Zhiyong Ye
2023-01-09 13:57 ` Kevin Wolf
0 siblings, 1 reply; 12+ messages in thread
From: Zhiyong Ye @ 2023-01-09 12:45 UTC (permalink / raw
To: Kevin Wolf, mreitz, qemu-block; +Cc: qemu-devel
Dear all,
Qemu provides powerful snapshot capabilities for different file formats.
But this is limited to the block backend being a file, and support is
not good enough when it is a block device. When creating snapshots based
on files, there is no need to specify the size of the snapshot image,
which can grow dynamically as the virtual machine is used. But block
devices are fixed in size at creation and cannot be dynamically grown at
a later time.
So is there any way to support snapshots when the block backend is a
block device?
Regards
Zhiyong
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-09 12:45 Questions about how block devices use snapshots Zhiyong Ye
@ 2023-01-09 13:57 ` Kevin Wolf
2023-01-11 7:55 ` Zhiyong Ye
2023-02-21 13:27 ` Zhiyong Ye
0 siblings, 2 replies; 12+ messages in thread
From: Kevin Wolf @ 2023-01-09 13:57 UTC (permalink / raw
To: Zhiyong Ye; +Cc: mreitz, qemu-block, qemu-devel
Am 09.01.2023 um 13:45 hat Zhiyong Ye geschrieben:
> Qemu provides powerful snapshot capabilities for different file
> formats. But this is limited to the block backend being a file, and
> support is not good enough when it is a block device. When creating
> snapshots based on files, there is no need to specify the size of the
> snapshot image, which can grow dynamically as the virtual machine is
> used. But block devices are fixed in size at creation and cannot be
> dynamically grown at a later time.
>
> So is there any way to support snapshots when the block backend is a
> block device?
In order to have snapshots, you need to have an image format like qcow2.
A qcow2 file can have a raw block device as its backing file, so even if
you store the overlay image on a filesystem, you have technically
snapshotted a block device. This may or may not be enough for your use
case.
It is also possible to store qcow2 files on block devices, though
depending on your requirements, it can get very tricky because then
you're responsible for making sure that there is always enough free
space on the block device.
So a second, still very simple, approach could be taking a second block
device that is a little bit larger than the virtual disk (for the qcow2
metadata) and use that as the external snapshot. Obviously, you require
a lot of disk space this way, because each snapshots needs to be able to
store the full image.
You could also use internal snapshots. In this case, you just need to
make sure that the block device is a lot larger than the virtual disk,
so that there is enough space left for storing the snapshots. At some
point it will be full.
And finally, for example if your block devices are actually LVs, you
could start resizing the block device dynmically as needed. This becomes
very complex quickly and you're on your own, but it is possible and has
been done by oVirt.
Kevin
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-09 13:57 ` Kevin Wolf
@ 2023-01-11 7:55 ` Zhiyong Ye
2023-01-11 14:32 ` Kevin Wolf
2023-02-21 13:27 ` Zhiyong Ye
1 sibling, 1 reply; 12+ messages in thread
From: Zhiyong Ye @ 2023-01-11 7:55 UTC (permalink / raw
To: Kevin Wolf; +Cc: mreitz, qemu-block, qemu-devel
Hi Kevin,
Thank you for your reply and detailed answers.
In my scenario is the iSCSI SAN environment. The network storage device
is connected to the physical machine via iSCSI, and LVM is used as the
middle layer between the storage device and the VM for storage
management and metadata synchronization. Every VM uses both raw and
qcow2 formats, with the system disk being qcow2 and the data disk being
raw. Therefore block devices need to support snapshot capability in both
raw and qcow2 store methods. In addition, snapshot images should also be
stored in iSCSI storage, which is a block device.
Both internal and external snapshots can implement snapshots of block
devices, but they both have their drawbacks when multiple snapshots are
required.
Internal snapshots can only be used in qcow2 format and do not require
additional creation of new block devices. As you said, the block device
has much more space than the virtual disk. There is no telling when disk
space will be full when creating multiple snapshots.
External snapshots require the creation of additional block devices to
store the overlay images, but it is not clear how much space needs to be
created. If the space is the same as the virtual disk, when there are
multiple snapshots it will be a serious waste of disk space, because
each time a new snapshot is created the previous one will become
read-only. However, if the disk space created is too small, the snapshot
data may not be stored when the disk space is full.
The problem with both is the uncertainty of the space size of the block
device at the time of creation. Of course, we can rely on lvm's resize
function to dynamically grow the space of the block device. But I think
this is more of a workaround.
It is mentioned in the Qemu docs page under "QEMU disk image utility"
that the qemu-img rebase can be used to perform a “diff” operation on
two disk images.
Say that base.img has been cloned as modified.img by copying it, and
that the modified.img guest has run so there are now some changes
compared to base.img. To construct a thin image called diff.qcow2 that
contains just the differences, do:
qemu-img create -f qcow2 -b modified.img diff.qcow2
qemu-img rebase -b base.img diff.qcow2
At this point, modified.img can be discarded, since base.img +
diff.qcow2 contains the same information.
Can this “diff” operation be used on snapshots of block devices? The
first snapshot is a copy of the original disk (to save space we can copy
only the data that has already been used), while the subsequent
snapshots are based on the diff of the previous snapshot, so that the
space required for the created block device is known at the time of the
snapshot.
Regards
Zhiyong
On 1/9/23 9:57 PM, Kevin Wolf wrote:
> Am 09.01.2023 um 13:45 hat Zhiyong Ye geschrieben:
>> Qemu provides powerful snapshot capabilities for different file
>> formats. But this is limited to the block backend being a file, and
>> support is not good enough when it is a block device. When creating
>> snapshots based on files, there is no need to specify the size of the
>> snapshot image, which can grow dynamically as the virtual machine is
>> used. But block devices are fixed in size at creation and cannot be
>> dynamically grown at a later time.
>>
>> So is there any way to support snapshots when the block backend is a
>> block device?
>
> In order to have snapshots, you need to have an image format like qcow2.
>
> A qcow2 file can have a raw block device as its backing file, so even if
> you store the overlay image on a filesystem, you have technically
> snapshotted a block device. This may or may not be enough for your use
> case.
>
> It is also possible to store qcow2 files on block devices, though
> depending on your requirements, it can get very tricky because then
> you're responsible for making sure that there is always enough free
> space on the block device.
>
> So a second, still very simple, approach could be taking a second block
> device that is a little bit larger than the virtual disk (for the qcow2
> metadata) and use that as the external snapshot. Obviously, you require
> a lot of disk space this way, because each snapshots needs to be able to
> store the full image.
>
> You could also use internal snapshots. In this case, you just need to
> make sure that the block device is a lot larger than the virtual disk,
> so that there is enough space left for storing the snapshots. At some
> point it will be full.
>
> And finally, for example if your block devices are actually LVs, you
> could start resizing the block device dynmically as needed. This becomes
> very complex quickly and you're on your own, but it is possible and has
> been done by oVirt.
>
> Kevin
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-11 7:55 ` Zhiyong Ye
@ 2023-01-11 14:32 ` Kevin Wolf
2023-01-11 16:21 ` Zhiyong Ye
0 siblings, 1 reply; 12+ messages in thread
From: Kevin Wolf @ 2023-01-11 14:32 UTC (permalink / raw
To: Zhiyong Ye; +Cc: mreitz, qemu-block, qemu-devel
Am 11.01.2023 um 08:55 hat Zhiyong Ye geschrieben:
> Hi Kevin,
>
> Thank you for your reply and detailed answers.
>
> In my scenario is the iSCSI SAN environment. The network storage device is
> connected to the physical machine via iSCSI, and LVM is used as the middle
> layer between the storage device and the VM for storage management and
> metadata synchronization. Every VM uses both raw and qcow2 formats, with the
> system disk being qcow2 and the data disk being raw. Therefore block devices
> need to support snapshot capability in both raw and qcow2 store methods. In
> addition, snapshot images should also be stored in iSCSI storage, which is a
> block device.
>
> Both internal and external snapshots can implement snapshots of block
> devices, but they both have their drawbacks when multiple snapshots are
> required.
>
> Internal snapshots can only be used in qcow2 format and do not require
> additional creation of new block devices. As you said, the block device has
> much more space than the virtual disk. There is no telling when disk space
> will be full when creating multiple snapshots.
>
> External snapshots require the creation of additional block devices to store
> the overlay images, but it is not clear how much space needs to be created.
> If the space is the same as the virtual disk, when there are multiple
> snapshots it will be a serious waste of disk space, because each time a new
> snapshot is created the previous one will become read-only. However, if the
> disk space created is too small, the snapshot data may not be stored when
> the disk space is full.
>
> The problem with both is the uncertainty of the space size of the block
> device at the time of creation. Of course, we can rely on lvm's resize
> function to dynamically grow the space of the block device. But I think this
> is more of a workaround.
Yes, this is why I said it's challenging. oVirt uses resizing of LVs to
achieve this, and it's still very complex. You need to decide yourself
if you think implementing the management software for this is worth it.
> It is mentioned in the Qemu docs page under "QEMU disk image utility" that
> the qemu-img rebase can be used to perform a “diff” operation on two disk
> images.
>
> Say that base.img has been cloned as modified.img by copying it, and that
> the modified.img guest has run so there are now some changes compared to
> base.img. To construct a thin image called diff.qcow2 that contains just the
> differences, do:
>
> qemu-img create -f qcow2 -b modified.img diff.qcow2
> qemu-img rebase -b base.img diff.qcow2
>
> At this point, modified.img can be discarded, since base.img + diff.qcow2
> contains the same information.
>
> Can this “diff” operation be used on snapshots of block devices? The first
> snapshot is a copy of the original disk (to save space we can copy only the
> data that has already been used), while the subsequent snapshots are based
> on the diff of the previous snapshot, so that the space required for the
> created block device is known at the time of the snapshot.
Yes, you can use raw block devices for both base.img and modified.img.
But of course, the result is still a qcow2 file that you need to store
somewhere.
Kevin
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-11 14:32 ` Kevin Wolf
@ 2023-01-11 16:21 ` Zhiyong Ye
2023-01-12 11:47 ` Kevin Wolf
0 siblings, 1 reply; 12+ messages in thread
From: Zhiyong Ye @ 2023-01-11 16:21 UTC (permalink / raw
To: Kevin Wolf; +Cc: mreitz, qemu-block, qemu-devel
Hi Kevin,
Can I ask again how base.img + diff.qcow2 can be re-merged into one
image via qemu-img or hmp command when modified.img is discarded?
Regards
Zhiyong
On 1/11/23 10:32 PM, Kevin Wolf wrote:
> Am 11.01.2023 um 08:55 hat Zhiyong Ye geschrieben:
>> Hi Kevin,
>>
>> Thank you for your reply and detailed answers.
>>
>> In my scenario is the iSCSI SAN environment. The network storage device is
>> connected to the physical machine via iSCSI, and LVM is used as the middle
>> layer between the storage device and the VM for storage management and
>> metadata synchronization. Every VM uses both raw and qcow2 formats, with the
>> system disk being qcow2 and the data disk being raw. Therefore block devices
>> need to support snapshot capability in both raw and qcow2 store methods. In
>> addition, snapshot images should also be stored in iSCSI storage, which is a
>> block device.
>>
>> Both internal and external snapshots can implement snapshots of block
>> devices, but they both have their drawbacks when multiple snapshots are
>> required.
>>
>> Internal snapshots can only be used in qcow2 format and do not require
>> additional creation of new block devices. As you said, the block device has
>> much more space than the virtual disk. There is no telling when disk space
>> will be full when creating multiple snapshots.
>>
>> External snapshots require the creation of additional block devices to store
>> the overlay images, but it is not clear how much space needs to be created.
>> If the space is the same as the virtual disk, when there are multiple
>> snapshots it will be a serious waste of disk space, because each time a new
>> snapshot is created the previous one will become read-only. However, if the
>> disk space created is too small, the snapshot data may not be stored when
>> the disk space is full.
>>
>> The problem with both is the uncertainty of the space size of the block
>> device at the time of creation. Of course, we can rely on lvm's resize
>> function to dynamically grow the space of the block device. But I think this
>> is more of a workaround.
>
> Yes, this is why I said it's challenging. oVirt uses resizing of LVs to
> achieve this, and it's still very complex. You need to decide yourself
> if you think implementing the management software for this is worth it.
>
>> It is mentioned in the Qemu docs page under "QEMU disk image utility" that
>> the qemu-img rebase can be used to perform a “diff” operation on two disk
>> images.
>>
>> Say that base.img has been cloned as modified.img by copying it, and that
>> the modified.img guest has run so there are now some changes compared to
>> base.img. To construct a thin image called diff.qcow2 that contains just the
>> differences, do:
>>
>> qemu-img create -f qcow2 -b modified.img diff.qcow2
>> qemu-img rebase -b base.img diff.qcow2
>>
>> At this point, modified.img can be discarded, since base.img + diff.qcow2
>> contains the same information.
>>
>> Can this “diff” operation be used on snapshots of block devices? The first
>> snapshot is a copy of the original disk (to save space we can copy only the
>> data that has already been used), while the subsequent snapshots are based
>> on the diff of the previous snapshot, so that the space required for the
>> created block device is known at the time of the snapshot.
>
> Yes, you can use raw block devices for both base.img and modified.img.
> But of course, the result is still a qcow2 file that you need to store
> somewhere.
>
> Kevin
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-11 16:21 ` Zhiyong Ye
@ 2023-01-12 11:47 ` Kevin Wolf
2023-01-13 8:30 ` Zhiyong Ye
0 siblings, 1 reply; 12+ messages in thread
From: Kevin Wolf @ 2023-01-12 11:47 UTC (permalink / raw
To: Zhiyong Ye; +Cc: mreitz, qemu-block, qemu-devel
Am 11.01.2023 um 17:21 hat Zhiyong Ye geschrieben:
> Hi Kevin,
>
> Can I ask again how base.img + diff.qcow2 can be re-merged into one image
> via qemu-img or hmp command when modified.img is discarded?
You can either use 'qemu-img commit' to copy all of the data from
diff.qcow2 back into base.img (this is probably what you want), or
'qemu-img rebase' to copy all of the data from base.img into diff.qcow2.
Kevin
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-12 11:47 ` Kevin Wolf
@ 2023-01-13 8:30 ` Zhiyong Ye
0 siblings, 0 replies; 12+ messages in thread
From: Zhiyong Ye @ 2023-01-13 8:30 UTC (permalink / raw
To: Kevin Wolf; +Cc: mreitz, qemu-block, qemu-devel
Hi Kevin,
Thank you for your patience and explanations. I learned a lot from our
discussions and thank you again for your help.
Regards
Zhiyong
On 1/12/23 7:47 PM, Kevin Wolf wrote:
> Am 11.01.2023 um 17:21 hat Zhiyong Ye geschrieben:
>> Hi Kevin,
>>
>> Can I ask again how base.img + diff.qcow2 can be re-merged into one image
>> via qemu-img or hmp command when modified.img is discarded?
>
> You can either use 'qemu-img commit' to copy all of the data from
> diff.qcow2 back into base.img (this is probably what you want), or
> 'qemu-img rebase' to copy all of the data from base.img into diff.qcow2.
>
> Kevin
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-01-09 13:57 ` Kevin Wolf
2023-01-11 7:55 ` Zhiyong Ye
@ 2023-02-21 13:27 ` Zhiyong Ye
2023-02-21 15:58 ` Kevin Wolf
1 sibling, 1 reply; 12+ messages in thread
From: Zhiyong Ye @ 2023-02-21 13:27 UTC (permalink / raw
To: Kevin Wolf; +Cc: mreitz, qemu-block, qemu-devel
Hi Kevin,
Sorry to bother you again.
I intend to use this approach for snapshots of block devices, which, as
you say, requires a lot of disk space to store snapshot data. So, to
save disk space, after each successful external snapshot creation, I
want to shrink the block device that stores the backing_file image to
the size that qcow2 data actually occupies, since it has become
read-only. But there is no way to get the actual size of qcow2 when it
is stored in a block device.
Qemu-img info can easily get the actual size of qcow2 when it is stored
in a file using the fstat function, but this will fail and return 0 for
block devices. Therefore, it is necessary to implement the method of
getting data occupancy inside qcow2. I think there may be two possible
ways to do this:
- Add a cluster count field @nb_clusters in the BDRVQcow2State for each
new cluster allocated and the actual size occupied by qcow2 is:
nb_clusters * cluster_size.
- Iterate through the refcount block to find the value with the largest
host offset, and this is the actual size occupied by qcow2.
Since I'm not very familiar with qcow2, may I ask if you have any advice
on getting the actual size when using qcow2?
Regards
Zhiyong
On 1/9/23 9:57 PM, Kevin Wolf wrote:
>
> So a second, still very simple, approach could be taking a second block
> device that is a little bit larger than the virtual disk (for the qcow2
> metadata) and use that as the external snapshot. Obviously, you require
> a lot of disk space this way, because each snapshots needs to be able to
> store the full image.
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-02-21 13:27 ` Zhiyong Ye
@ 2023-02-21 15:58 ` Kevin Wolf
2023-02-23 7:35 ` Zhiyong Ye
0 siblings, 1 reply; 12+ messages in thread
From: Kevin Wolf @ 2023-02-21 15:58 UTC (permalink / raw
To: Zhiyong Ye; +Cc: mreitz, qemu-block, qemu-devel
Am 21.02.2023 um 14:27 hat Zhiyong Ye geschrieben:
>
> Hi Kevin,
>
> Sorry to bother you again.
>
> I intend to use this approach for snapshots of block devices, which, as you
> say, requires a lot of disk space to store snapshot data. So, to save disk
> space, after each successful external snapshot creation, I want to shrink
> the block device that stores the backing_file image to the size that qcow2
> data actually occupies, since it has become read-only. But there is no way
> to get the actual size of qcow2 when it is stored in a block device.
>
> Qemu-img info can easily get the actual size of qcow2 when it is stored in a
> file using the fstat function, but this will fail and return 0 for block
> devices. Therefore, it is necessary to implement the method of getting data
> occupancy inside qcow2. I think there may be two possible ways to do this:
>
> - Add a cluster count field @nb_clusters in the BDRVQcow2State for each new
> cluster allocated and the actual size occupied by qcow2 is: nb_clusters *
> cluster_size.
> - Iterate through the refcount block to find the value with the largest host
> offset, and this is the actual size occupied by qcow2.
>
> Since I'm not very familiar with qcow2, may I ask if you have any advice on
> getting the actual size when using qcow2?
I think what you need is the 'image-end-offset' field from 'qemu-img
check --output=json'.
Kevin
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-02-21 15:58 ` Kevin Wolf
@ 2023-02-23 7:35 ` Zhiyong Ye
2023-02-23 11:39 ` Kevin Wolf
0 siblings, 1 reply; 12+ messages in thread
From: Zhiyong Ye @ 2023-02-23 7:35 UTC (permalink / raw
To: Kevin Wolf; +Cc: mreitz, qemu-block, qemu-devel
Hi Kevin,
Thank you for your reply and this method works.
May I ask if this 'image-end-offset' field can be shown in the qemu-img
info too? Because it is also a very useful information whether qcow2 is
placed on a file or a block device.
Regards
Zhiyong
On 2/21/23 11:58 PM, Kevin Wolf wrote:
> Am 21.02.2023 um 14:27 hat Zhiyong Ye geschrieben:
>>
>> Hi Kevin,
>>
>> Sorry to bother you again.
>>
>> I intend to use this approach for snapshots of block devices, which, as you
>> say, requires a lot of disk space to store snapshot data. So, to save disk
>> space, after each successful external snapshot creation, I want to shrink
>> the block device that stores the backing_file image to the size that qcow2
>> data actually occupies, since it has become read-only. But there is no way
>> to get the actual size of qcow2 when it is stored in a block device.
>>
>> Qemu-img info can easily get the actual size of qcow2 when it is stored in a
>> file using the fstat function, but this will fail and return 0 for block
>> devices. Therefore, it is necessary to implement the method of getting data
>> occupancy inside qcow2. I think there may be two possible ways to do this:
>>
>> - Add a cluster count field @nb_clusters in the BDRVQcow2State for each new
>> cluster allocated and the actual size occupied by qcow2 is: nb_clusters *
>> cluster_size.
>> - Iterate through the refcount block to find the value with the largest host
>> offset, and this is the actual size occupied by qcow2.
>>
>> Since I'm not very familiar with qcow2, may I ask if you have any advice on
>> getting the actual size when using qcow2?
>
> I think what you need is the 'image-end-offset' field from 'qemu-img
> check --output=json'.
>
> Kevin
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-02-23 7:35 ` Zhiyong Ye
@ 2023-02-23 11:39 ` Kevin Wolf
2023-02-23 11:47 ` Zhiyong Ye
0 siblings, 1 reply; 12+ messages in thread
From: Kevin Wolf @ 2023-02-23 11:39 UTC (permalink / raw
To: Zhiyong Ye; +Cc: mreitz, qemu-block, qemu-devel
Am 23.02.2023 um 08:35 hat Zhiyong Ye geschrieben:
> Hi Kevin,
>
> Thank you for your reply and this method works.
>
> May I ask if this 'image-end-offset' field can be shown in the qemu-img info
> too? Because it is also a very useful information whether qcow2 is placed on
> a file or a block device.
The only way to know the largest offset is by looking at all the
metadata in qcow2. 'qemu-img info' is supposed to be very fast, so we
don't do that there. 'qemu-img check' already looks at all metadata, so
we have the number readily available there.
Kevin
> On 2/21/23 11:58 PM, Kevin Wolf wrote:
> > Am 21.02.2023 um 14:27 hat Zhiyong Ye geschrieben:
> > >
> > > Hi Kevin,
> > >
> > > Sorry to bother you again.
> > >
> > > I intend to use this approach for snapshots of block devices, which, as you
> > > say, requires a lot of disk space to store snapshot data. So, to save disk
> > > space, after each successful external snapshot creation, I want to shrink
> > > the block device that stores the backing_file image to the size that qcow2
> > > data actually occupies, since it has become read-only. But there is no way
> > > to get the actual size of qcow2 when it is stored in a block device.
> > >
> > > Qemu-img info can easily get the actual size of qcow2 when it is stored in a
> > > file using the fstat function, but this will fail and return 0 for block
> > > devices. Therefore, it is necessary to implement the method of getting data
> > > occupancy inside qcow2. I think there may be two possible ways to do this:
> > >
> > > - Add a cluster count field @nb_clusters in the BDRVQcow2State for each new
> > > cluster allocated and the actual size occupied by qcow2 is: nb_clusters *
> > > cluster_size.
> > > - Iterate through the refcount block to find the value with the largest host
> > > offset, and this is the actual size occupied by qcow2.
> > >
> > > Since I'm not very familiar with qcow2, may I ask if you have any advice on
> > > getting the actual size when using qcow2?
> >
> > I think what you need is the 'image-end-offset' field from 'qemu-img
> > check --output=json'.
> >
> > Kevin
> >
>
^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: Questions about how block devices use snapshots
2023-02-23 11:39 ` Kevin Wolf
@ 2023-02-23 11:47 ` Zhiyong Ye
0 siblings, 0 replies; 12+ messages in thread
From: Zhiyong Ye @ 2023-02-23 11:47 UTC (permalink / raw
To: Kevin Wolf; +Cc: mreitz, qemu-block, qemu-devel
Hi Kevin,
Thank you for your patience and explanations.
Thanks again!
Zhiyong
On 2/23/23 7:39 PM, Kevin Wolf wrote:
> Am 23.02.2023 um 08:35 hat Zhiyong Ye geschrieben:
>> Hi Kevin,
>>
>> Thank you for your reply and this method works.
>>
>> May I ask if this 'image-end-offset' field can be shown in the qemu-img info
>> too? Because it is also a very useful information whether qcow2 is placed on
>> a file or a block device.
>
> The only way to know the largest offset is by looking at all the
> metadata in qcow2. 'qemu-img info' is supposed to be very fast, so we
> don't do that there. 'qemu-img check' already looks at all metadata, so
> we have the number readily available there.
>
> Kevin
>
>> On 2/21/23 11:58 PM, Kevin Wolf wrote:
>>> Am 21.02.2023 um 14:27 hat Zhiyong Ye geschrieben:
>>>>
>>>> Hi Kevin,
>>>>
>>>> Sorry to bother you again.
>>>>
>>>> I intend to use this approach for snapshots of block devices, which, as you
>>>> say, requires a lot of disk space to store snapshot data. So, to save disk
>>>> space, after each successful external snapshot creation, I want to shrink
>>>> the block device that stores the backing_file image to the size that qcow2
>>>> data actually occupies, since it has become read-only. But there is no way
>>>> to get the actual size of qcow2 when it is stored in a block device.
>>>>
>>>> Qemu-img info can easily get the actual size of qcow2 when it is stored in a
>>>> file using the fstat function, but this will fail and return 0 for block
>>>> devices. Therefore, it is necessary to implement the method of getting data
>>>> occupancy inside qcow2. I think there may be two possible ways to do this:
>>>>
>>>> - Add a cluster count field @nb_clusters in the BDRVQcow2State for each new
>>>> cluster allocated and the actual size occupied by qcow2 is: nb_clusters *
>>>> cluster_size.
>>>> - Iterate through the refcount block to find the value with the largest host
>>>> offset, and this is the actual size occupied by qcow2.
>>>>
>>>> Since I'm not very familiar with qcow2, may I ask if you have any advice on
>>>> getting the actual size when using qcow2?
>>>
>>> I think what you need is the 'image-end-offset' field from 'qemu-img
>>> check --output=json'.
>>>
>>> Kevin
>>>
>>
>
^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2023-02-23 11:48 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-01-09 12:45 Questions about how block devices use snapshots Zhiyong Ye
2023-01-09 13:57 ` Kevin Wolf
2023-01-11 7:55 ` Zhiyong Ye
2023-01-11 14:32 ` Kevin Wolf
2023-01-11 16:21 ` Zhiyong Ye
2023-01-12 11:47 ` Kevin Wolf
2023-01-13 8:30 ` Zhiyong Ye
2023-02-21 13:27 ` Zhiyong Ye
2023-02-21 15:58 ` Kevin Wolf
2023-02-23 7:35 ` Zhiyong Ye
2023-02-23 11:39 ` Kevin Wolf
2023-02-23 11:47 ` Zhiyong Ye
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.