From: "Michael S. Tsirkin" <mst@redhat.com>
To: Babis Chalios <bchalios@amazon.es>
Cc: virtio-comment@lists.oasis-open.org,
virtio-dev@lists.oasis-open.org, "Cali,
Marco" <xmarcalx@amazon.co.uk>, "Graf (AWS),
Alexander" <graf@amazon.de>,
"Jason A. Donenfeld" <Jason@zx2c4.com>,
aams@amazon.de
Subject: [virtio-comment] Re: [virtio-dev] [PATCH RFC 3/3] rng: leak detection support
Date: Thu, 2 Nov 2023 07:51:49 -0400 [thread overview]
Message-ID: <20231102074645-mutt-send-email-mst@kernel.org> (raw)
In-Reply-To: <23ca2091-efd8-498d-8ed6-9ac4fb905f1d@amazon.es>
On Thu, Nov 02, 2023 at 12:38:28PM +0100, Babis Chalios wrote:
>
> On 2/11/23 12:20, Michael S. Tsirkin wrote:
> >
> > On Thu, Sep 28, 2023 at 08:16:11PM +0200, Babis Chalios wrote:
> > >
> > > On 27/9/23 23:47, Michael S. Tsirkin wrote:
> > > > On Wed, Sep 27, 2023 at 12:43:20PM +0200, Babis Chalios wrote:
> > > > > On 22/9/23 18:01, Michael S. Tsirkin wrote:
> > > > > > On Fri, Sep 22, 2023 at 05:40:50PM +0200, Babis Chalios wrote:
> > > > > > > On 22/9/23 17:06, Michael S. Tsirkin wrote:
> > > > > > > > On Tue, Sep 19, 2023 at 12:11:37PM +0200, Babis Chalios wrote:
> > > > > > > > > On 19/9/23 12:01, Michael S. Tsirkin wrote:
> > > > > > > > > > On Tue, Sep 19, 2023 at 09:32:08AM +0200, Babis Chalios wrote:
> > > > > > > > > > > Resending to fix e-mail formatting issues (sorry for the spam)
> > > > > > > > > > >
> > > > > > > > > > > On 18/9/23 18:30, Babis Chalios wrote:
> > > > > > > > > > > > > > > > Yes, that's what the driver does now in the RFC patch.
> > > > > > > > > > > > > > > > However, this just
> > > > > > > > > > > > > > > > decreases
> > > > > > > > > > > > > > > > the race window, it doesn't eliminate it. If a third
> > > > > > > > > > > > > > > > leak event happens it
> > > > > > > > > > > > > > > > might not
> > > > > > > > > > > > > > > > find any buffers to use:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 1. available buffers to queue 1-X
> > > > > > > > > > > > > > > > 2. available buffers to queue X
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 3. poll queue X
> > > > > > > > > > > > > > > > 4. used buffers in queue X <- leak event 1 will
> > > > > > > > > > > > > > > > use buffers in X
> > > > > > > > > > > > > > > > 5. avail buffers in queue X
> > > > > > > > > > > > > > > > 6. poll queue 1-X <- leak event 2 will
> > > > > > > > > > > > > > > > use buffers in 1-X
> > > > > > > > > > > > > > > > 7. used buffers in queue 1-X
> > > > > > > > > > > > > > > > 8. avail buffers in queue 1-X
> > > > > > > > > > > > > > > > <- leak event 3 (it
> > > > > > > > > > > > > > > > needs buffers in X, race with step 5)
> > > > > > > > > > > > > > > > 9. goto 3
> > > > > > > > > > > > > > > I don't get it. we added buffers in step 5.
> > > > > > > > > > > > > > What if the leak event 3 arrives before step 5 had time to
> > > > > > > > > > > > > > actually add the
> > > > > > > > > > > > > > buffers in X and make
> > > > > > > > > > > > > > them visible to the device?
> > > > > > > > > > > > > Then it will see a single event in 1-X instead of two events. A leak is
> > > > > > > > > > > > > a leak though, I don't see does it matter how many triggered.
> > > > > > > > > > > > >
> > > > > > > > > > > So the scenario I have in mind is the following:
> > > > > > > > > > >
> > > > > > > > > > > (Epoch here is terminology that I used in the Linux RFC. It is a value
> > > > > > > > > > > maintained by random.c
> > > > > > > > > > > that changes every time a leak event happens).
> > > > > > > > > > >
> > > > > > > > > > > 1. add buffers to 1-X
> > > > > > > > > > > 2. add buffers to X
> > > > > > > > > > > 3. poll queue X
> > > > > > > > > > > 4. vcpu 0: get getrandom() entropy and cache epoch value
> > > > > > > > > > > 5. Device: First snapshot, uses buffers in X
> > > > > > > > > > > 6. vcpu 1: sees used buffers
> > > > > > > > > > > 7. Device: Second snapshot, uses buffers in 1-X
> > > > > > > > > > > 8. vcpu 0: getrandom() observes new epoch value & caches it
> > > > > > > > > > > 9. Device: Third snapshot, no buffers in either queue, (vcpu 1 from step 6
> > > > > > > > > > > has not yet finished adding new buffers).
> > > > > > > > > > > 10. vcpu 1 adds new buffer in X
> > > > > > > > > > > 11. vcpu 0: getrandom() will not see new epoch and gets stale entropy.
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > In this succession of events, when the third snapshot will happen, the
> > > > > > > > > > > device won't find
> > > > > > > > > > > any buffers in either queue, so it won't increase the RNG epoch value. So,
> > > > > > > > > > > any entropy
> > > > > > > > > > > gathered after step 8 will be the same across all snapshots. Am I missing
> > > > > > > > > > > something?
> > > > > > > > > > >
> > > > > > > > > > > Cheers,
> > > > > > > > > > > Babis
> > > > > > > > > > >
> > > > > > > > > > Yes but notice how this is followed by:
> > > > > > > > > >
> > > > > > > > > > 12. vcpu 1: sees used buffers in 1-X
> > > > > > > > > >
> > > > > > > > > > Driver can notify getrandom I guess?
> > > > > > > > > It could, but then we have the exact race condition that VMGENID had,
> > > > > > > > > userspace has already consumed stale entropy and there's nothing we
> > > > > > > > > can do about that.
> > > > > > > > >
> > > > > > > > > Although this is indeed a corner case, it feels like it beats the purpose
> > > > > > > > > of having the hardware update directly userspace (via copy on leak).
> > > > > > > > >
> > > > > > > > > How do you feel about the proposal a couple of emails back? It looks to
> > > > > > > > > me that it avoids completely the race condition.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Babis
> > > > > > > > It does. The problem of course is that this means that e.g.
> > > > > > > > taking a snapshot of a guest that is stuck won't work well.
> > > > > > > That is true, but does it matter? The intention of the proposal
> > > > > > > is that if it is not safe to take snapshots (i.e. no buffers in the
> > > > > > > queue) don't take snapshots.
> > > > > > >
> > > > > > > > I have been thinking of adding MAP/UNMAP descriptors for
> > > > > > > > a while now. Thus it will be possible to modify
> > > > > > > > userspace memory without consuming buffers.
> > > > > > > > Would something like this solve the problem?
> > > > > > > I am not familiar with MAP/UNMAP descriptors. Is there
> > > > > > > a link where I can read about them?
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Babis
> > > > > > Heh no I just came up with the name. Will write up in a couple
> > > > > > of days, but the idea is that driver does get_user_pages,
> > > > > > adds buffer to queue, and device will remember the address
> > > > > > and change that memory on a snapshot. If there are buffers
> > > > > > in the queue it will also use these to tell driver,
> > > > > > but if there are no buffers then it won't.
> > > > > That sounds like a nice mechanism. However in our case the page
> > > > > holding the counter that gets increased by the hardware is a kernel
> > > > > page.
> > > > >
> > > > > The reason for that is that things other than us (virtio-rng) might
> > > > > want to notify for leak events. For example, I think that Jason
> > > > > intended to use this mechanism to periodically notify user-space
> > > > > PRNGs that they need to reseed.
> > > > >
> > > > > Cheers,
> > > > > Babis
> > > > Now I'm lost.
> > > > when you write, e.g.:
> > > > 4. vcpu 0: get getrandom() entropy and cache epoch value
> > > > how does vcpu access the epoch?
> > > The kernel provides a user space API to map a pointer to the epoch
> > > value. User space then caches its value and checks it every time it
> > > needs to make sure that no entropy leak has happened before using
> > > cached kernel entropy.
> > >
> > > virtio-rng driver adds a copy on leak command to the queue for
> > > increasing this value (that's what we are speaking about in this thread).
> > > But other systems might want to report "leaks", such as random.c
> > > itself.
> > >
> > > Cheers,
> > > Babis
> >
> > This idea would be fine but I don't see how it's so different
> > from VMGENID. Care to explain?
> >
> It is different in that the memory is owned by the guest kernel, not
> the hardware. In this case, random.c maintains it. This allows, systems
> other than the hardware, e.g. virtio-rng to notify about entropy leak
> events. For example, random.c itself can periodically do that (I think
> Jason had that use-case in mind).
No sure I understand. Example?
>
> The fact that we as well mmap that memory to the user-space is just
> for giving user space a mechanism that allows it to know when it needs
> to reseed its PRNGs. We _could_ have done the same with VMGENID,
> but the fact that, in this case, the underlying physical memory is owned
> by the ACPI device blocks us from letting other systems sending
> notifications
> as well.
>
> Maybe it makes sense to have this discussion in the patch we sent for
> Linux on LKML [1]?
>
> Cheers,
> Babis
>
> [1] https://lore.kernel.org/lkml/20230823090107.65749-3-bchalios@amazon.es/T/#mb1242999d8296169d9a4ee1a0805005633ec146a
This publicly archived list offers a means to provide input to the
OASIS Virtual I/O Device (VIRTIO) TC.
In order to verify user consent to the Feedback License terms and
to minimize spam in the list archive, subscription is required
before posting.
Subscribe: virtio-comment-subscribe@lists.oasis-open.org
Unsubscribe: virtio-comment-unsubscribe@lists.oasis-open.org
List help: virtio-comment-help@lists.oasis-open.org
List archive: https://lists.oasis-open.org/archives/virtio-comment/
Feedback License: https://www.oasis-open.org/who/ipr/feedback_license.pdf
List Guidelines: https://www.oasis-open.org/policies-guidelines/mailing-lists
Committee: https://www.oasis-open.org/committees/virtio/
Join OASIS: https://www.oasis-open.org/join/
next prev parent reply other threads:[~2023-11-02 11:52 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-21 16:30 [virtio-comment] [PATCH RFC 0/3] virtio-rng based entropy leak reporting Michael S. Tsirkin
2022-11-21 16:30 ` [virtio-comment] [PATCH RFC 1/3] rng: move to a file of its own Michael S. Tsirkin
2022-11-21 16:30 ` [virtio-comment] [PATCH RFC 2/3] rng: be specific about the virtqueue Michael S. Tsirkin
2022-11-21 16:30 ` [virtio-dev] [PATCH RFC 3/3] rng: leak detection support Michael S. Tsirkin
2022-11-25 12:41 ` [virtio-dev] " Babis Chalios
2022-12-12 10:10 ` Babis Chalios
2023-01-11 13:57 ` Babis Chalios
[not found] ` <0abb7518-16e0-4227-bfe1-a29bd27124e8@amazon.es>
2023-09-12 21:05 ` [virtio-comment] Re: [virtio-dev] " Michael S. Tsirkin
[not found] ` <e738e7f0-dd13-44db-806c-543746aa6967@amazon.es>
2023-09-13 9:37 ` Michael S. Tsirkin
[not found] ` <d7d17e9f-e6ad-4106-a408-99281382a93f@amazon.es>
2023-09-18 12:41 ` Michael S. Tsirkin
[not found] ` <e75d9667-2ea0-44d0-a26f-fa3d5fe05651@amazon.es>
2023-09-18 13:58 ` Michael S. Tsirkin
[not found] ` <a22ba2cb-9b03-481a-b66e-0b78fbfb21f0@amazon.es>
2023-09-18 14:05 ` Michael S. Tsirkin
[not found] ` <4ab1b10e-4524-4b6c-a3df-863360fd225d@amazon.es>
[not found] ` <c2e7076b-b07e-43cd-a912-9314d742a678@amazon.es>
2023-09-19 10:01 ` Michael S. Tsirkin
[not found] ` <4c6d6710-43c8-402b-9ae6-d2f4cafbe9a7@amazon.es>
2023-09-22 15:06 ` Michael S. Tsirkin
[not found] ` <c0c5b159-997f-4ddf-ab3e-1c749ad93e10@amazon.es>
2023-09-22 16:01 ` Michael S. Tsirkin
[not found] ` <19967fde-8893-4fe8-9f9a-7ce49bf38772@amazon.es>
2023-09-27 21:47 ` Michael S. Tsirkin
[not found] ` <4d22d699-b9f3-4d67-940a-8f11bcd29819@amazon.es>
[not found] ` <5f548d39-2ae4-45a8-9dba-49a2c693dae4@amazon.es>
2023-10-13 13:38 ` Michael S. Tsirkin
2023-11-02 11:20 ` Michael S. Tsirkin
[not found] ` <23ca2091-efd8-498d-8ed6-9ac4fb905f1d@amazon.es>
2023-11-02 11:51 ` Michael S. Tsirkin [this message]
2023-11-02 11:25 ` Michael S. Tsirkin
2023-01-12 7:02 ` [virtio-dev] Re: [PATCH RFC 0/3] virtio-rng based entropy leak reporting Michael S. Tsirkin
2023-01-16 11:39 ` Babis Chalios
[not found] ` <CAHmME9ry2fss2gsbPs2zVJkY=8Cdeae0XFD9FzCVnW67Xy3thA@mail.gmail.com>
2023-01-16 18:11 ` [virtio-comment] " Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20231102074645-mutt-send-email-mst@kernel.org \
--to=mst@redhat.com \
--cc=Jason@zx2c4.com \
--cc=aams@amazon.de \
--cc=bchalios@amazon.es \
--cc=graf@amazon.de \
--cc=virtio-comment@lists.oasis-open.org \
--cc=virtio-dev@lists.oasis-open.org \
--cc=xmarcalx@amazon.co.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).