From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
To: qemu-block@nongnu.org
Cc: qemu-devel@nongnu.org, vsementsov@virtuozzo.com,
eblake@redhat.com, mreitz@redhat.com, kwolf@redhat.com,
pbonzini@redhat.com
Subject: [PATCH v4 10/32] block/nbd: simplify waking of nbd_co_establish_connection()
Date: Thu, 10 Jun 2021 13:07:40 +0300 [thread overview]
Message-ID: <20210610100802.5888-11-vsementsov@virtuozzo.com> (raw)
In-Reply-To: <20210610100802.5888-1-vsementsov@virtuozzo.com>
Instead of managing connect_bh, bh_ctx, and wait_connect fields, we
can use a single link to the waiting coroutine with proper mutex
protection.
So new logic is:
nbd_co_establish_connection() sets wait_co under the mutex, releases
the mutex, then yield()s. Note that wait_co may be scheduled by the
thread immediately after unlocking the mutex. Still, the main thread
(or iothread) will not reach the code for entering the coroutine until
the yield(), so we are safe.
connect_thread_func() and nbd_co_establish_connection_cancel() do
the following to handle wait_co:
Under the mutex, if thr->wait_co is not NULL, make it NULL and
schedule it. This way, we avoid scheduling the coroutine twice.
Still scheduling is a bit different:
In connect_thread_func() we can just call aio_co_wake under mutex,
after commit
[async: the main AioContext is only "current" if under the BQL]
we are sure that aio_co_wake() will not try to acquire the aio context
and do qemu_aio_coroutine_enter() but simply schedule the coroutine by
aio_co_schedule().
nbd_co_establish_connection_cancel() will be called from non-coroutine
context in further patch and will be able to go through
qemu_aio_coroutine_enter() path of aio_co_wake(). So keep current
behavior of waking the coroutine after the critical section.
Also, this commit reduces the dependence of
nbd_co_establish_connection() on the internals of bs (we now use a
generic pointer to the coroutine, instead of direct use of
s->connection_co). This is a step towards splitting the connection
API out of nbd.c.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
---
block/nbd.c | 55 +++++++++++++++--------------------------------------
1 file changed, 15 insertions(+), 40 deletions(-)
diff --git a/block/nbd.c b/block/nbd.c
index fdfb1ff7a1..653af62940 100644
--- a/block/nbd.c
+++ b/block/nbd.c
@@ -87,12 +87,6 @@ typedef enum NBDConnectThreadState {
typedef struct NBDConnectThread {
/* Initialization constants */
SocketAddress *saddr; /* address to connect to */
- /*
- * Bottom half to schedule on completion. Scheduled only if bh_ctx is not
- * NULL
- */
- QEMUBHFunc *bh_func;
- void *bh_opaque;
/*
* Result of last attempt. Valid in FAIL and SUCCESS states.
@@ -101,10 +95,15 @@ typedef struct NBDConnectThread {
QIOChannelSocket *sioc;
Error *err;
- /* state and bh_ctx are protected by mutex */
QemuMutex mutex;
+ /* All further fields are protected by mutex */
NBDConnectThreadState state; /* current state of the thread */
- AioContext *bh_ctx; /* where to schedule bh (NULL means don't schedule) */
+
+ /*
+ * wait_co: if non-NULL, which coroutine to wake in
+ * nbd_co_establish_connection() after yield()
+ */
+ Coroutine *wait_co;
} NBDConnectThread;
typedef struct BDRVNBDState {
@@ -138,7 +137,6 @@ typedef struct BDRVNBDState {
char *x_dirty_bitmap;
bool alloc_depth;
- bool wait_connect;
NBDConnectThread *connect_thread;
} BDRVNBDState;
@@ -370,15 +368,6 @@ static bool nbd_client_connecting_wait(BDRVNBDState *s)
return qatomic_load_acquire(&s->state) == NBD_CLIENT_CONNECTING_WAIT;
}
-static void connect_bh(void *opaque)
-{
- BDRVNBDState *state = opaque;
-
- assert(state->wait_connect);
- state->wait_connect = false;
- aio_co_wake(state->connection_co);
-}
-
static void nbd_init_connect_thread(BDRVNBDState *s)
{
s->connect_thread = g_new(NBDConnectThread, 1);
@@ -386,8 +375,6 @@ static void nbd_init_connect_thread(BDRVNBDState *s)
*s->connect_thread = (NBDConnectThread) {
.saddr = QAPI_CLONE(SocketAddress, s->saddr),
.state = CONNECT_THREAD_NONE,
- .bh_func = connect_bh,
- .bh_opaque = s,
};
qemu_mutex_init(&s->connect_thread->mutex);
@@ -427,11 +414,9 @@ static void *connect_thread_func(void *opaque)
switch (thr->state) {
case CONNECT_THREAD_RUNNING:
thr->state = ret < 0 ? CONNECT_THREAD_FAIL : CONNECT_THREAD_SUCCESS;
- if (thr->bh_ctx) {
- aio_bh_schedule_oneshot(thr->bh_ctx, thr->bh_func, thr->bh_opaque);
-
- /* play safe, don't reuse bh_ctx on further connection attempts */
- thr->bh_ctx = NULL;
+ if (thr->wait_co) {
+ aio_co_wake(thr->wait_co);
+ thr->wait_co = NULL;
}
break;
case CONNECT_THREAD_RUNNING_DETACHED:
@@ -485,20 +470,14 @@ nbd_co_establish_connection(BlockDriverState *bs, Error **errp)
abort();
}
- thr->bh_ctx = qemu_get_current_aio_context();
+ thr->wait_co = qemu_coroutine_self();
qemu_mutex_unlock(&thr->mutex);
-
/*
* We are going to wait for connect-thread finish, but
* nbd_client_co_drain_begin() can interrupt.
- *
- * Note that wait_connect variable is not visible for connect-thread. It
- * doesn't need mutex protection, it used only inside home aio context of
- * bs.
*/
- s->wait_connect = true;
qemu_coroutine_yield();
qemu_mutex_lock(&thr->mutex);
@@ -553,23 +532,19 @@ static void nbd_co_establish_connection_cancel(BlockDriverState *bs)
{
BDRVNBDState *s = bs->opaque;
NBDConnectThread *thr = s->connect_thread;
- bool wake = false;
+ Coroutine *wait_co = NULL;
qemu_mutex_lock(&thr->mutex);
if (thr->state == CONNECT_THREAD_RUNNING) {
/* We can cancel only in running state, when bh is not yet scheduled */
- thr->bh_ctx = NULL;
- if (s->wait_connect) {
- s->wait_connect = false;
- wake = true;
- }
+ wait_co = g_steal_pointer(&thr->wait_co);
}
qemu_mutex_unlock(&thr->mutex);
- if (wake) {
- aio_co_wake(s->connection_co);
+ if (wait_co) {
+ aio_co_wake(wait_co);
}
}
--
2.29.2
next prev parent reply other threads:[~2021-06-10 10:13 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-10 10:07 [PATCH v4 00/32] block/nbd: rework client connection Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 01/32] co-queue: drop extra coroutine_fn marks Vladimir Sementsov-Ogievskiy
2021-06-10 17:22 ` Eric Blake
2021-06-10 17:37 ` Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 02/32] block/nbd: fix channel object leak Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 03/32] block/nbd: fix how state is cleared on nbd_open() failure paths Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 04/32] block/nbd: connect_thread_func(): do qio_channel_set_delay(false) Vladimir Sementsov-Ogievskiy
2021-06-10 18:37 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 05/32] qemu-sockets: introduce socket_address_parse_named_fd() Vladimir Sementsov-Ogievskiy
2021-06-11 13:22 ` Eric Blake
2021-06-11 14:10 ` Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 06/32] block/nbd: call socket_address_parse_named_fd() in advance Vladimir Sementsov-Ogievskiy
2021-06-11 13:54 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 07/32] block/nbd: ensure ->connection_thread is always valid Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 08/32] block/nbd: nbd_client_handshake(): fix leak of s->ioc Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 09/32] block/nbd: BDRVNBDState: drop unused connect_err and connect_status Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` Vladimir Sementsov-Ogievskiy [this message]
2021-06-11 14:06 ` [PATCH v4 10/32] block/nbd: simplify waking of nbd_co_establish_connection() Eric Blake
2021-06-10 10:07 ` [PATCH v4 11/32] block/nbd: drop thr->state Vladimir Sementsov-Ogievskiy
2021-06-11 14:25 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 12/32] block/nbd: bs-independent interface for nbd_co_establish_connection() Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 13/32] block/nbd: make nbd_co_establish_connection_cancel() bs-independent Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 14/32] block/nbd: rename NBDConnectThread to NBDClientConnection Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 15/32] block/nbd: introduce nbd_client_connection_new() Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 16/32] block/nbd: introduce nbd_client_connection_release() Vladimir Sementsov-Ogievskiy
2021-06-11 14:28 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 17/32] nbd: move connection code from block/nbd to nbd/client-connection Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 18/32] nbd/client-connection: use QEMU_LOCK_GUARD Vladimir Sementsov-Ogievskiy
2021-06-11 14:31 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 19/32] nbd/client-connection: add possibility of negotiation Vladimir Sementsov-Ogievskiy
2021-06-11 15:07 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 20/32] nbd/client-connection: implement connection retry Vladimir Sementsov-Ogievskiy
2021-06-11 15:12 ` Eric Blake
2021-11-22 16:30 ` Eric Blake
2021-11-22 17:17 ` Vladimir Sementsov-Ogievskiy
2021-11-22 21:51 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 21/32] nbd/client-connection: shutdown connection on release Vladimir Sementsov-Ogievskiy
2021-06-11 15:27 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 22/32] block/nbd: split nbd_handle_updated_info out of nbd_client_handshake() Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 23/32] block/nbd: use negotiation of NBDClientConnection Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 24/32] block/nbd: don't touch s->sioc in nbd_teardown_connection() Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 25/32] block/nbd: drop BDRVNBDState::sioc Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 26/32] nbd/client-connection: return only one io channel Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 27/32] block-coroutine-wrapper: allow non bdrv_ prefix Vladimir Sementsov-Ogievskiy
2021-06-10 10:07 ` [PATCH v4 28/32] block/nbd: split nbd_co_do_establish_connection out of nbd_reconnect_attempt Vladimir Sementsov-Ogievskiy
2021-06-11 15:29 ` Eric Blake
2021-06-10 10:07 ` [PATCH v4 29/32] nbd/client-connection: add option for non-blocking connection attempt Vladimir Sementsov-Ogievskiy
2021-06-10 10:08 ` [PATCH v4 30/32] block/nbd: reuse nbd_co_do_establish_connection() in nbd_open() Vladimir Sementsov-Ogievskiy
2021-06-10 10:08 ` [PATCH v4 31/32] block/nbd: add nbd_client_connected() helper Vladimir Sementsov-Ogievskiy
2021-06-10 10:08 ` [PATCH v4 32/32] block/nbd: safer transition to receiving request Vladimir Sementsov-Ogievskiy
2021-06-11 15:55 ` [PATCH v4 00/32] block/nbd: rework client connection Eric Blake
2021-06-11 17:23 ` Vladimir Sementsov-Ogievskiy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210610100802.5888-11-vsementsov@virtuozzo.com \
--to=vsementsov@virtuozzo.com \
--cc=eblake@redhat.com \
--cc=kwolf@redhat.com \
--cc=mreitz@redhat.com \
--cc=pbonzini@redhat.com \
--cc=qemu-block@nongnu.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).