From: Fabiano Rosas <farosas@suse.de>
To: "Daniel P. Berrangé" <berrange@redhat.com>
Cc: qemu-devel@nongnu.org, armbru@redhat.com,
Peter Xu <peterx@redhat.com>, Claudio Fontana <cfontana@suse.de>,
Jim Fehlig <jfehlig@suse.com>
Subject: Re: [PATCH 2/9] migration: Fix file migration with fdset
Date: Wed, 08 May 2024 17:45:47 -0300 [thread overview]
Message-ID: <87a5l0kp6s.fsf@suse.de> (raw)
In-Reply-To: <ZjsxE9p4CsfZQK8Y@redhat.com>
Daniel P. Berrangé <berrange@redhat.com> writes:
> On Fri, Apr 26, 2024 at 11:20:35AM -0300, Fabiano Rosas wrote:
>> When the migration using the "file:" URI was implemented, I don't
>> think any of us noticed that if you pass in a file name with the
>> format "/dev/fdset/N", this allows a file descriptor to be passed in
>> to QEMU and that behaves just like the "fd:" URI. So the "file:"
>> support has been added without regard for the fdset part and we got
>> some things wrong.
>>
>> The first issue is that we should not truncate the migration file if
>> we're allowing an fd + offset. We need to leave the file contents
>> untouched.
>>
>> The second issue is that there's an expectation that QEMU removes the
>> fd after the migration has finished. That's what the "fd:" code
>> does. Otherwise a second migration on the same VM could attempt to
>> provide an fdset with the same name and QEMU would reject it.
>>
>> We can fix the first issue by detecting when we're using the fdset
>> vs. the plain file name. This requires storing the fdset_id
>> somewhere. We can then use this stored fdset_id to do cleanup at the
>> end and also fix the second issue.
>
> The use of /dev/fdset is supposed to be transparent to code in
> QEMU, so modifying migration to learn about FD sets to do manual
> cleanup is breaking that API facade.
>
> IMHO the transparency of the design points towards the mgmt app
> calling 'remove-fd' set after migration has started, in order
> that a later migraiton can use the same fdset name.
I got this slightly wrong, QEMU doesn't reject the creation of the
fdset, it just reuses the old one and adds the new fd to it. That is
somewhat worse because then we'd choose the wrong fd when migrating. But
I guess we could just require the management layer to do proper
management of the fds/fdset.
>
> Ideally the truncation issue needs to be transparent too.
>
> Rather than detecting use of fdset, we can not use O_TRUNC
> at all. Instead we can call ftruncate(fd, offset), which
> should work in both normal and fdset scenarios.
>
Good idea.
>>
>> Fixes: 385f510df5 ("migration: file URI offset")
>> Signed-off-by: Fabiano Rosas <farosas@suse.de>
>> ---
>> migration/file.c | 48 ++++++++++++++++++++++++++++++++++++++++++++++--
>> 1 file changed, 46 insertions(+), 2 deletions(-)
>>
>> diff --git a/migration/file.c b/migration/file.c
>> index ab18ba505a..8f30999400 100644
>> --- a/migration/file.c
>> +++ b/migration/file.c
>> @@ -10,6 +10,7 @@
>> #include "qemu/cutils.h"
>> #include "qemu/error-report.h"
>> #include "qapi/error.h"
>> +#include "qapi/qapi-commands-misc.h"
>> #include "channel.h"
>> #include "file.h"
>> #include "migration.h"
>> @@ -23,6 +24,7 @@
>>
>> static struct FileOutgoingArgs {
>> char *fname;
>> + int64_t fdset_id;
>> } outgoing_args;
>>
>> /* Remove the offset option from @filespec and return it in @offsetp. */
>> @@ -44,10 +46,39 @@ int file_parse_offset(char *filespec, uint64_t *offsetp, Error **errp)
>> return 0;
>> }
>>
>> +static void file_remove_fdset(void)
>> +{
>> + if (outgoing_args.fdset_id != -1) {
>> + qmp_remove_fd(outgoing_args.fdset_id, false, -1, NULL);
>> + outgoing_args.fdset_id = -1;
>> + }
>> +}
>> +
>> +static bool file_parse_fdset(const char *filename, int64_t *fdset_id,
>> + Error **errp)
>> +{
>> + const char *fdset_id_str;
>> +
>> + *fdset_id = -1;
>> +
>> + if (!strstart(filename, "/dev/fdset/", &fdset_id_str)) {
>> + return true;
>> + }
>> +
>> + *fdset_id = qemu_parse_fd(fdset_id_str);
>> + if (*fdset_id == -1) {
>> + error_setg_errno(errp, EINVAL, "Could not parse fdset %s", fdset_id_str);
>> + return false;
>> + }
>> +
>> + return true;
>> +}
>> +
>> void file_cleanup_outgoing_migration(void)
>> {
>> g_free(outgoing_args.fname);
>> outgoing_args.fname = NULL;
>> + file_remove_fdset();
>> }
>>
>> bool file_send_channel_create(gpointer opaque, Error **errp)
>> @@ -81,11 +112,24 @@ void file_start_outgoing_migration(MigrationState *s,
>> g_autofree char *filename = g_strdup(file_args->filename);
>> uint64_t offset = file_args->offset;
>> QIOChannel *ioc;
>> + int flags = O_CREAT | O_WRONLY;
>>
>> trace_migration_file_outgoing(filename);
>>
>> - fioc = qio_channel_file_new_path(filename, O_CREAT | O_WRONLY | O_TRUNC,
>> - 0600, errp);
>> + if (!file_parse_fdset(filename, &outgoing_args.fdset_id, errp)) {
>> + return;
>> + }
>> +
>> + /*
>> + * Only truncate if it's QEMU opening the file. If an fd has been
>> + * passed in the file will already contain data written by the
>> + * management layer.
>> + */
>> + if (outgoing_args.fdset_id == -1) {
>> + flags |= O_TRUNC;
>> + }
>> +
>> + fioc = qio_channel_file_new_path(filename, flags, 0600, errp);
>> if (!fioc) {
>> return;
>> }
>> --
>> 2.35.3
>>
>
> With regards,
> Daniel
next prev parent reply other threads:[~2024-05-08 20:46 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-26 14:20 [PATCH 0/9] migration/mapped-ram: Add direct-io support Fabiano Rosas
2024-04-26 14:20 ` [PATCH 1/9] monitor: Honor QMP request for fd removal immediately Fabiano Rosas
2024-05-03 16:02 ` Peter Xu
2024-05-16 21:46 ` Fabiano Rosas
2024-05-08 7:17 ` Daniel P. Berrangé
2024-05-16 22:00 ` Fabiano Rosas
2024-05-17 7:33 ` Daniel P. Berrangé
2024-04-26 14:20 ` [PATCH 2/9] migration: Fix file migration with fdset Fabiano Rosas
2024-05-03 16:23 ` Peter Xu
2024-05-03 19:56 ` Fabiano Rosas
2024-05-03 21:04 ` Peter Xu
2024-05-03 21:31 ` Fabiano Rosas
2024-05-03 21:56 ` Peter Xu
2024-05-08 8:02 ` Daniel P. Berrangé
2024-05-08 12:49 ` Peter Xu
2024-05-08 8:00 ` Daniel P. Berrangé
2024-05-08 20:45 ` Fabiano Rosas [this message]
2024-04-26 14:20 ` [PATCH 3/9] tests/qtest/migration: Fix file migration offset check Fabiano Rosas
2024-05-03 16:47 ` Peter Xu
2024-05-03 20:36 ` Fabiano Rosas
2024-05-03 21:08 ` Peter Xu
2024-05-08 8:10 ` Daniel P. Berrangé
2024-04-26 14:20 ` [PATCH 4/9] migration: Add direct-io parameter Fabiano Rosas
2024-04-26 14:33 ` Markus Armbruster
2024-05-03 18:05 ` Peter Xu
2024-05-03 20:49 ` Fabiano Rosas
2024-05-03 21:16 ` Peter Xu
2024-05-14 14:10 ` Markus Armbruster
2024-05-14 17:57 ` Fabiano Rosas
2024-05-15 7:17 ` Markus Armbruster
2024-05-15 12:51 ` Fabiano Rosas
2024-05-08 8:25 ` Daniel P. Berrangé
2024-04-26 14:20 ` [PATCH 5/9] migration/multifd: Add direct-io support Fabiano Rosas
2024-05-03 18:29 ` Peter Xu
2024-05-03 20:54 ` Fabiano Rosas
2024-05-03 21:18 ` Peter Xu
2024-05-08 8:27 ` Daniel P. Berrangé
2024-04-26 14:20 ` [PATCH 6/9] tests/qtest/migration: Add tests for file migration with direct-io Fabiano Rosas
2024-05-03 18:38 ` Peter Xu
2024-05-03 21:05 ` Fabiano Rosas
2024-05-03 21:25 ` Peter Xu
2024-05-08 8:34 ` Daniel P. Berrangé
2024-04-26 14:20 ` [PATCH 7/9] monitor: fdset: Match against O_DIRECT Fabiano Rosas
2024-05-03 18:53 ` Peter Xu
2024-05-03 21:19 ` Fabiano Rosas
2024-05-03 22:16 ` Peter Xu
2024-04-26 14:20 ` [PATCH 8/9] migration: Add support for fdset with multifd + file Fabiano Rosas
2024-05-08 8:53 ` Daniel P. Berrangé
2024-05-08 18:23 ` Peter Xu
2024-05-08 20:39 ` Fabiano Rosas
2024-05-09 8:08 ` Daniel P. Berrangé
2024-05-17 22:43 ` Fabiano Rosas
2024-05-18 8:36 ` Daniel P. Berrangé
2024-04-26 14:20 ` [PATCH 9/9] tests/qtest/migration: Add a test for mapped-ram with passing of fds Fabiano Rosas
2024-05-08 8:56 ` Daniel P. Berrangé
2024-05-02 20:01 ` [PATCH 0/9] migration/mapped-ram: Add direct-io support Peter Xu
2024-05-02 20:34 ` Fabiano Rosas
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87a5l0kp6s.fsf@suse.de \
--to=farosas@suse.de \
--cc=armbru@redhat.com \
--cc=berrange@redhat.com \
--cc=cfontana@suse.de \
--cc=jfehlig@suse.com \
--cc=peterx@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).