From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, vsementsov@virtuozzo.com,
jsnow@redhat.com, mreitz@redhat.com, kwolf@redhat.com
Subject: [PATCH v5 00/35] block: publish backup-top filter
Date: Thu, 10 Jun 2021 14:25:43 +0300 [thread overview]
Message-ID: <20210610112618.127378-1-vsementsov@virtuozzo.com> (raw)
Hi all!
v5: rebase on master
25: fix pylint complain 'Unnecessary "else" after "return"' [John]
25-26: rebased on moved machine.py
use Dict[str, object] typing instead of Dict[str, Any] [John]
add John's r-b, keep Max's r-b
Patches without r-b: 9, 10, 19, 20
v4: rebase on Kevin's block branch
v3: Keep x-perf.copy-range backup option.
So, additional function is added to set copy_range in block-copy after creation.
And if we do so, it's better to set "compress" option same way instead of handling
x-deprecated-compress option.
8: rebased on not yet remove @perf argument, keep r-b
11, 21, 23: small related rebase conflicts, keep r-b
9, 10, 19: new
20: drop x-deprecated-compress option, drop r-b
24: fix typo
34: fix qmp( , **{}) to qmp( , {})
also, add a lot of Max's r-bs, thanks!!!
v2:
01-02: new
03: don't bother with supporting empty child: we should never have such
at this point
05: add comment
06: keep checking conflict with global
add realized_set_allowed to qdev_prop_drive_iothread
07: improve cbw_cbw() name
improve commit message
10: rebased on unchanged backup_calculate_cluster_size(). keep r-b CHECK ME
12: new
13: drop extra bdrv_unref()
18: add compress local variable
add comment about x-deprecated-compress
19: new, replacement for "[PATCH 17/21] block/block-copy: switch to fully set bitmap by default"
22: improve qapi documentation
23-33: test: a lot of refactoring
We have image fleecing scheme to export point-in-time state of active
disk (iotest 222):
backup(sync=none)
┌───────────────────────────────────────┐
▼ │
┌────────────┐ ┌────────────────┐ backing ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘ └────────────────┘ └─────────────┘
▲
┌────────────┐ │
│ guest blk │ ──────────────────────────────────────────────┘
└────────────┘
Actually, backup job inserts a backup-top filter, so in detail it looks
like:
backup(sync=none)
┌───────────────────────────────────────┐
▼ │
┌────────────┐ ┌────────────────┐ backing ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘ └────────────────┘ └─────────────┘
▲ ▲
│ target │
│ │
┌────────────┐ ┌────────────────┐ backing │
│ guest blk │ ──▶ │ backup-top │ ───────────────────────┘
└────────────┘ └────────────────┘
And job does nothing here. In a new blockdev world user is intended to
operate on node level, and insert/remove filters by hand. Let's get rid
of job in the scheme:
┌────────────┐ ┌────────────────┐ backing ┌─────────────┐
│ NBD export │ ─── │ temp qcow2 img │ ───────────────────▶ │ active disk │
└────────────┘ └────────────────┘ └─────────────┘
▲ ▲
│ target │
│ │
┌────────────┐ ┌────────────────┐ backing │
│ guest blk │ ──▶ │ backup-top │ ───────────────────────┘
└────────────┘ └────────────────┘
The series prepares qom-set to make possible inserting filters above
root node (patches 03-06), rename backup-top to copy-before-write, do
other preparations for publishing the filter, and finally publish it,
add qapi interface and test new fleecing scheme in 222 (first, some
good test refactoring).
Vladimir Sementsov-Ogievskiy (35):
block: rename bdrv_replace_child to bdrv_replace_child_tran
block: comment graph-modifying function not updating permissions
block: introduce bdrv_replace_child_bs()
block: introduce blk_replace_bs
qdev-properties: PropertyInfo: add realized_set_allowed field
qdev: allow setting drive property for realized device
block: rename backup-top to copy-before-write
block-copy: always set BDRV_REQ_SERIALISING flag
block/block-copy: introduce block_copy_set_copy_opts()
block/backup: set copy_range and compress after filter insertion
block/backup: move cluster size calculation to block-copy
block/copy-before-write: relax permission requirements when no parents
block/copy-before-write: drop extra bdrv_unref on failure path
block/copy-before-write: use file child instead of backing
block/copy-before-write: bdrv_cbw_append(): replace child at last
block/copy-before-write: introduce cbw_init()
block/copy-before-write: cbw_init(): rename variables
block/copy-before-write: cbw_init(): use file child after attaching
block/copy-before-write: bdrv_cbw_append(): drop unused compress arg
block/copy-before-write: cbw_init(): use options
block/copy-before-write: initialize block-copy bitmap
block/block-copy: make setting progress optional
block/copy-before-write: make public block driver
qapi: publish copy-before-write filter
python/qemu/machine.py: refactor _qemu_args()
python/qemu/machine: QEMUMachine: improve qmp() method
iotests.py: VM: add own __enter__ method
iotests/222: fix pylint and mypy complains
iotests/222: constantly use single quotes for strings
iotests: move 222 to tests/image-fleecing
iotests.py: hmp_qemu_io: support qdev
iotests/image-fleecing: proper source device
iotests/image-fleecing: rename tgt_node
iotests/image-fleecing: prepare for adding new test-case
iotests/image-fleecing: add test-case for copy-before-write filter
qapi/block-core.json | 30 ++-
block/{backup-top.h => copy-before-write.h} | 25 +-
include/block/block-copy.h | 6 +-
include/block/block.h | 2 +
include/hw/qdev-properties.h | 1 +
include/sysemu/block-backend.h | 1 +
block.c | 53 +++-
block/backup-top.c | 253 -------------------
block/backup.c | 116 ++-------
block/block-backend.c | 8 +
block/block-copy.c | 148 ++++++++---
block/copy-before-write.c | 256 ++++++++++++++++++++
hw/core/qdev-properties-system.c | 43 +++-
hw/core/qdev-properties.c | 6 +-
MAINTAINERS | 4 +-
block/meson.build | 2 +-
python/qemu/machine/machine.py | 30 ++-
tests/qemu-iotests/222 | 159 ------------
tests/qemu-iotests/222.out | 67 -----
tests/qemu-iotests/283 | 35 ++-
tests/qemu-iotests/283.out | 4 +-
tests/qemu-iotests/297 | 2 +-
tests/qemu-iotests/iotests.py | 9 +-
tests/qemu-iotests/tests/image-fleecing | 192 +++++++++++++++
tests/qemu-iotests/tests/image-fleecing.out | 139 +++++++++++
25 files changed, 911 insertions(+), 680 deletions(-)
rename block/{backup-top.h => copy-before-write.h} (56%)
delete mode 100644 block/backup-top.c
create mode 100644 block/copy-before-write.c
delete mode 100755 tests/qemu-iotests/222
delete mode 100644 tests/qemu-iotests/222.out
create mode 100755 tests/qemu-iotests/tests/image-fleecing
create mode 100644 tests/qemu-iotests/tests/image-fleecing.out
--
2.29.2
next reply other threads:[~2021-06-10 11:34 UTC|newest]
Thread overview: 36+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-10 11:25 Vladimir Sementsov-Ogievskiy [this message]
2021-06-10 11:25 ` [PATCH v5 01/35] block: rename bdrv_replace_child to bdrv_replace_child_tran Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 02/35] block: comment graph-modifying function not updating permissions Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 03/35] block: introduce bdrv_replace_child_bs() Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 04/35] block: introduce blk_replace_bs Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 05/35] qdev-properties: PropertyInfo: add realized_set_allowed field Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 06/35] qdev: allow setting drive property for realized device Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 07/35] block: rename backup-top to copy-before-write Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 08/35] block-copy: always set BDRV_REQ_SERIALISING flag Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 09/35] block/block-copy: introduce block_copy_set_copy_opts() Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 10/35] block/backup: set copy_range and compress after filter insertion Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 11/35] block/backup: move cluster size calculation to block-copy Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 12/35] block/copy-before-write: relax permission requirements when no parents Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 13/35] block/copy-before-write: drop extra bdrv_unref on failure path Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 14/35] block/copy-before-write: use file child instead of backing Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 15/35] block/copy-before-write: bdrv_cbw_append(): replace child at last Vladimir Sementsov-Ogievskiy
2021-06-10 11:25 ` [PATCH v5 16/35] block/copy-before-write: introduce cbw_init() Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 17/35] block/copy-before-write: cbw_init(): rename variables Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 18/35] block/copy-before-write: cbw_init(): use file child after attaching Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 19/35] block/copy-before-write: bdrv_cbw_append(): drop unused compress arg Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 20/35] block/copy-before-write: cbw_init(): use options Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 21/35] block/copy-before-write: initialize block-copy bitmap Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 22/35] block/block-copy: make setting progress optional Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 23/35] block/copy-before-write: make public block driver Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 24/35] qapi: publish copy-before-write filter Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 25/35] python/qemu/machine.py: refactor _qemu_args() Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 26/35] python/qemu/machine: QEMUMachine: improve qmp() method Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 27/35] iotests.py: VM: add own __enter__ method Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 28/35] iotests/222: fix pylint and mypy complains Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 29/35] iotests/222: constantly use single quotes for strings Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 30/35] iotests: move 222 to tests/image-fleecing Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 31/35] iotests.py: hmp_qemu_io: support qdev Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 32/35] iotests/image-fleecing: proper source device Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 33/35] iotests/image-fleecing: rename tgt_node Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 34/35] iotests/image-fleecing: prepare for adding new test-case Vladimir Sementsov-Ogievskiy
2021-06-10 11:26 ` [PATCH v5 35/35] iotests/image-fleecing: add test-case for copy-before-write filter Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210610112618.127378-1-vsementsov@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=jsnow@redhat.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).