From: Hanna Czenczek <hreitz@redhat.com>
To: qemu-devel@nongnu.org, virtio-fs@redhat.com
Cc: Hanna Czenczek <hreitz@redhat.com>,
Stefan Hajnoczi <stefanha@redhat.com>,
German Maglione <gmaglione@redhat.com>,
Anton Kuchin <antonkuchin@yandex-team.ru>,
Juan Quintela <quintela@redhat.com>,
"Michael S . Tsirkin" <mst@redhat.com>,
Stefano Garzarella <sgarzare@redhat.com>
Subject: [Virtio-fs] [PATCH 0/4] vhost-user-fs: Internal migration
Date: Tue, 11 Apr 2023 17:05:11 +0200 [thread overview]
Message-ID: <20230411150515.14020-1-hreitz@redhat.com> (raw)
RFC:
https://lists.nongnu.org/archive/html/qemu-devel/2023-03/msg04263.html
Hi,
Patch 2 of this series adds new vhost methods (only for vhost-user at
this point) for transferring the back-end’s internal state to/from qemu
during migration, so that this state can be stored in the migration
stream. (This is what we call “internal migration”, because the state
is internally available to qemu; this is in contrast to “external
migration”, which Anton is working on, where the back-end’s state is
handled by the back-end itself without involving qemu.)
For this, the state is handled as a binary blob by qemu, and it is
transferred over a pipe that is established via a new vhost method.
Patch 3 adds two high-level helper functions to (A) fetch any vhost
back-end’s internal state and store it in a migration stream (a
`QEMUFile`), and (B) load such state from a migrations stream and send
it to a vhost back-end. These build on the low-level interface
introduced in patch 2.
Patch 4 then uses these functions to implement internal migration for
vhost-user-fs. Note that this of course depends on support in the
back-end (virtiofsd), which is not yet ready.
Finally, patch 1 fixes a bug around migrating vhost-user devices: To
enable/disable logging[1], the VHOST_F_LOG_ALL feature must be
set/cleared, via the SET_FEATURES call. Another, technically unrelated,
feature exists, VHOST_USER_F_PROTOCOL_FEATURES, which indicates support
for vhost-user protocol features. Naturally, qemu wants to keep that
other feature enabled, so it will set it (when possible) in every
SET_FEATURES call. However, a side effect of setting
VHOST_USER_F_PROTOCOL_FEATURES is that all vrings are disabled. This
causes any enabling (done at the start of migration) or disabling (done
on the source after a cancelled/failed migration) of logging to make the
back-end hang. Without patch 1, therefore, starting a migration will
have any vhost-user back-end that supports both VHOST_F_LOG_ALL and
VHOST_USER_F_PROTOCOL_FEATURES immediately hang completely, and unless
execution is transferred to the destination, it will continue to hang.
[1] Logging here means logging writes to guest memory pages in a dirty
bitmap so that these dirty pages are flushed to the destination. qemu
cannot monitor the back-end’s writes to guest memory, so the back-end
has to do so itself, and log its writes in a dirty bitmap shared with
qemu.
Changes in v1 compared to the RFC:
- Patch 1 added
- Patch 2: Interface is different, now uses a pipe instead of shared
memory (as suggested by Stefan); also, this is now a generic
vhost-user interface, and not just for vhost-user-fs
- Patches 3 and 4: Because this is now supposed to be a generic
migration method for vhost-user back-ends, most of the migration code
has been moved from vhost-user-fs.c to vhost.c so it can be shared
between different back-ends. The vhost-user-fs code is now a rather
thin wrapper around the common code.
- Note also (as suggested by Anton) that the back-end’s migration
state is now in a subsection, and that it is technically optional.
“Technically” means that with this series, it is always used (unless
the back-end doesn’t support migration, in which case migration is
just blocked), but Anton’s series for external migration would make
it optional. (I.e., the subsection would be skipped for external
migration, and mandatorily included for internal migration.)
Hanna Czenczek (4):
vhost: Re-enable vrings after setting features
vhost-user: Interface for migration state transfer
vhost: Add high-level state save/load functions
vhost-user-fs: Implement internal migration
include/hw/virtio/vhost-backend.h | 24 +++
include/hw/virtio/vhost.h | 124 +++++++++++++++
hw/virtio/vhost-user-fs.c | 101 +++++++++++-
hw/virtio/vhost-user.c | 147 ++++++++++++++++++
hw/virtio/vhost.c | 246 ++++++++++++++++++++++++++++++
5 files changed, 641 insertions(+), 1 deletion(-)
--
2.39.1
next reply other threads:[~2023-04-11 15:05 UTC|newest]
Thread overview: 93+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-11 15:05 Hanna Czenczek [this message]
2023-04-11 15:05 ` [Virtio-fs] [PATCH 1/4] vhost: Re-enable vrings after setting features Hanna Czenczek
2023-04-12 10:55 ` German Maglione
2023-04-12 12:18 ` Hanna Czenczek
2023-04-12 20:51 ` Stefan Hajnoczi
2023-04-13 7:17 ` Maxime Coquelin
2023-04-13 8:19 ` Hanna Czenczek
2023-04-13 11:03 ` Stefan Hajnoczi
2023-04-13 14:24 ` Anton Kuchin
2023-04-13 15:48 ` Michael S. Tsirkin
2023-04-13 11:03 ` Stefan Hajnoczi
2023-04-13 17:32 ` Hanna Czenczek
2023-04-13 13:19 ` Michael S. Tsirkin
2023-04-11 15:05 ` [Virtio-fs] [PATCH 2/4] vhost-user: Interface for migration state transfer Hanna Czenczek
2023-04-12 21:06 ` Stefan Hajnoczi
2023-04-13 9:24 ` Hanna Czenczek
2023-04-13 11:38 ` Stefan Hajnoczi
2023-04-13 17:55 ` Hanna Czenczek
2023-04-13 20:42 ` Stefan Hajnoczi
2023-04-14 15:17 ` Eugenio Perez Martin
2023-04-17 15:18 ` Stefan Hajnoczi
2023-04-17 18:55 ` Eugenio Perez Martin
2023-04-17 19:08 ` Stefan Hajnoczi
2023-04-17 19:11 ` Eugenio Perez Martin
2023-04-17 19:46 ` Stefan Hajnoczi
2023-04-18 10:09 ` Eugenio Perez Martin
2023-04-19 10:45 ` Hanna Czenczek
2023-04-19 10:57 ` Stefan Hajnoczi
2023-04-13 10:14 ` Eugenio Perez Martin
2023-04-13 11:07 ` Stefan Hajnoczi
2023-04-13 17:31 ` Hanna Czenczek
2023-04-17 15:12 ` Stefan Hajnoczi
2023-04-19 10:47 ` Hanna Czenczek
2023-04-17 18:37 ` Eugenio Perez Martin
2023-04-17 15:38 ` Stefan Hajnoczi
2023-04-17 19:09 ` Eugenio Perez Martin
2023-04-17 19:33 ` Stefan Hajnoczi
2023-04-18 8:09 ` Eugenio Perez Martin
2023-04-18 17:59 ` Stefan Hajnoczi
2023-04-18 18:31 ` Eugenio Perez Martin
2023-04-18 20:40 ` Stefan Hajnoczi
2023-04-20 13:27 ` Eugenio Pérez
2023-05-08 19:12 ` Stefan Hajnoczi
2023-05-09 6:31 ` Eugenio Perez Martin
2023-05-09 9:01 ` Hanna Czenczek
2023-05-09 15:26 ` Eugenio Perez Martin
2023-04-19 10:57 ` Hanna Czenczek
2023-04-19 11:10 ` Stefan Hajnoczi
2023-04-19 11:15 ` Hanna Czenczek
2023-04-19 11:24 ` Stefan Hajnoczi
2023-04-17 17:14 ` Stefan Hajnoczi
2023-04-17 19:06 ` Eugenio Perez Martin
2023-04-17 19:20 ` Stefan Hajnoczi
2023-04-18 7:54 ` Eugenio Perez Martin
2023-04-19 11:10 ` Hanna Czenczek
2023-04-19 11:21 ` Stefan Hajnoczi
2023-04-19 11:24 ` Hanna Czenczek
2023-04-20 13:29 ` Eugenio Pérez
2023-05-08 20:10 ` Stefan Hajnoczi
2023-05-09 6:45 ` Eugenio Perez Martin
2023-05-09 15:09 ` Stefan Hajnoczi
2023-05-09 15:35 ` Eugenio Perez Martin
2023-05-09 17:33 ` Stefan Hajnoczi
2023-04-20 10:44 ` Eugenio Pérez
2023-04-13 8:50 ` Eugenio Perez Martin
2023-04-13 9:25 ` Hanna Czenczek
2023-04-11 15:05 ` [Virtio-fs] [PATCH 3/4] vhost: Add high-level state save/load functions Hanna Czenczek
2023-04-12 21:14 ` Stefan Hajnoczi
2023-04-13 9:04 ` Hanna Czenczek
2023-04-13 11:22 ` Stefan Hajnoczi
2023-04-11 15:05 ` [Virtio-fs] [PATCH 4/4] vhost-user-fs: Implement internal migration Hanna Czenczek
2023-04-12 21:00 ` [Virtio-fs] [PATCH 0/4] vhost-user-fs: Internal migration Stefan Hajnoczi
2023-04-13 8:20 ` Hanna Czenczek
2023-04-13 16:11 ` Michael S. Tsirkin
2023-04-13 17:53 ` Hanna Czenczek
2023-05-04 16:05 ` Hanna Czenczek
2023-05-04 21:14 ` Stefan Hajnoczi
2023-05-05 9:03 ` Hanna Czenczek
2023-05-05 9:51 ` Hanna Czenczek
2023-05-05 14:26 ` Eugenio Perez Martin
2023-05-05 14:37 ` Hanna Czenczek
2023-05-08 17:00 ` Hanna Czenczek
2023-05-08 17:51 ` Eugenio Perez Martin
2023-05-08 19:31 ` Eugenio Perez Martin
2023-05-09 8:59 ` Hanna Czenczek
2023-05-09 15:30 ` Stefan Hajnoczi
2023-05-09 15:43 ` Eugenio Perez Martin
2023-05-05 9:53 ` Eugenio Perez Martin
2023-05-05 12:51 ` Hanna Czenczek
2023-05-08 21:10 ` Stefan Hajnoczi
2023-05-09 8:53 ` Hanna Czenczek
2023-05-09 14:53 ` Stefan Hajnoczi
2023-05-09 15:41 ` Stefan Hajnoczi
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230411150515.14020-1-hreitz@redhat.com \
--to=hreitz@redhat.com \
--cc=antonkuchin@yandex-team.ru \
--cc=gmaglione@redhat.com \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
--cc=quintela@redhat.com \
--cc=sgarzare@redhat.com \
--cc=stefanha@redhat.com \
--cc=virtio-fs@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).