From: Alexander Gordeev <alexander.gordeev@opensynergy.com>
To: "Alexandre Courbot" <acourbot@chromium.org>,
virtio-dev@lists.oasis-open.org,
"Keiichi Watanabe" <keiichiw@chromium.org>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Enrico Granata" <egranata@google.com>,
"Gustavo Padovan" <gustavo.padovan@collabora.com>,
"Peter Griffin" <peter.griffin@linaro.org>,
"Hans Verkuil" <hverkuil@xs4all.nl>,
"Nicolas Dufresne" <nicolas.dufresne@collabora.com>,
"Albert Esteve" <aesteve@redhat.com>,
"Enric Balletbo i Serra" <eballetb@redhat.com>,
"Daniel Almeida" <daniel.almeida@collabora.com>
Cc: Linux Media Mailing List <linux-media@vger.kernel.org>
Subject: Re: [virtio-dev] [FYI] virtio-media published
Date: Fri, 15 Dec 2023 18:06:09 +0100 [thread overview]
Message-ID: <565caa81-c265-4986-80b5-5a0e0ebbfe6d@opensynergy.com> (raw)
In-Reply-To: <CAPBb6MUB9GTEOCCkvVK9+ukjuYr9x6X8Kv-XSHT1tNd_=ZCCfA@mail.gmail.com>
Hello Alexandre,
On 08.12.23 07:46, Alexandre Courbot wrote:
> Hello everyone,
>
> This is just a heads-up to announce the initial publication of virtio-media:
>
> https://github.com/Gnurou/virtio-media
>
> The name has changed slightly, but this project is essentially the
> refinement and continuation of my virtio-v4l2 proposal. The reason for
> moving forward with this is, to be candid, the unclear direction and
> lack of progress regarding the future of virtio-video, and the
> non-consensual way in which its development has been, let's say
> "redirected".
Great! I like the new name. The old one was confusing IMO. I'm going to
have a closer look later.
I'd be happy to clarify the current progress and the future direction of
the virtio-video. I'd like to make these major changes in the next spec
draft:
1. Represent the capabilities and parameters with TLVs. Basically I'd
like to take the data format from VIDIOC_G_EXT_CTRLS/VIDIOC_S_EXT_CTRLS,
flatten it by replacing the pointers with the actual data (this would be
then in fact a sequence of TLVs). Then use this data format when getting
capabilities (all of the VIDIOC_ENUM* and VIDIOC_QUERY* in a single
call), getting and setting parameters (parameters + controls in V4L2
terms). Here I'd like to reuse the control ids and value definitions as
much as possible. Maybe this data presentation could be used in V4L2
later. I think it would be digestible, as it is an extension of the
extended controls. Indeed it was not clear, I even thought device trees
would be a great solution at some point (see [1] for the rationale), but
several people convinced me, that's not going to be merged in the kernel.
BTW this activity was influenced by your comment in [2]:
> Beware of not repeating the same mistake as v5 (all controls under one
> big structure).
2. Make the input and output resource queues explicit by turning them
into virtio queues. It became clear, that after making the commands non
blocking, some of the queue control moves to these internal resource
queues. Then I thought: why make another queue implementation, when we
already have one? So I think it makes sense to turn them into virtio
queues. I still would like to keep the delayed responses over the event
queue because I like to have this ordering of the events. Otherwise the
DRAIN has to be done in some other way. Also I remember the comment by
Bartłomiej Grzesik in [3]:
> I remember having a few thoughts on how it could be solved and I
> think that removing the need to block those descriptors is the best
> approach in my opinion. One would argue that preallocating descriptors
> for this purpose or splitting the command queue to input, output and
> control might be a viable solution or per stream. However it would
> only delay the issue in time or could cause other streams to "starve".
So I'd like to try both approaches at once.
I hope to finish the next draft in 3-4 weeks and publish in January.
Hopefully the virtio release will be done and folks will have some time
for a review.
Also I've been busy updating the virtio-video driver. First I decided to
merge all the existing patches. We have a lot of not yet published
changes internally, also you and your team made a lot of changes to the
driver. I went through all of these, merged everything carefully,
learned about some of the problems, that you encountered. This was a big
chunk of work and now it is finished, see [4]. Still working on other
driver updates.
The first problem was about making a distinction between guest pages and
exported buffers. The final solution in your patches AFAIU is to map
VB2_MEMORY_USERPTR to guest pages and VB2_MEMORY_DMABUF to the exported
objects because this is the only hint, that we have from REQBUFS. I
think this is not good enough, because dmabufs can be of different
types: it could be an exported object or, for example, one created with
udmabuf. I don't think there is a way to provide this information to the
device until the first buffer comes, right? So the spec should be
changed in this regard. I think it would be enough to add a requirement,
that all the buffers belonging to a queue must be of the same type.
Another problem is setting the DMA masks and other such stuff. Our
current approach is to register the parent device, which has all the
related fields properly set as it is done in virtio-gpu, see [5].
Overall I'd like to highlight the fact, that you and your team already
have a lot of influence on the virtio-video direction. I was very busy
with investigations trying to select the right solution and merging the
patches, but I think this only makes the whole thing better. :) So your
comments are very welcome.
> The repo above contains the specification in the README file
> (presented in a more informal way than the virtio specification), as
> well as a guest V4L2 driver able to pass v4l2-compliance when proxying
> the vivid/vicodec virtual devices or an actual UVC camera using the
> crosvm V4L2 proxy device [1]. As of now it is basically
> feature-complete and offers everything that virtio-video is supposed
> to eventually do. I am considering adding support for multiple devices
> and requests to allow more complex camera setups.
>
> Since the initial proposal has been rejected I have no intent to push
> this forward for merging in the virtio specification, so this will
> live out of the official spec. However, it is likely that we will soon
> switch to this solution for video support in ChromeOS and possibly
> other Google projects with a similar need for a stable virtualization
> solution for media devices.
>
> Anyone who is interested in using this for their project, or with
> specific requests, is welcome to contact me or open issues on the
> Github project.
>
> Cheers,
> Alex.
>
> [1] https://chromium-review.googlesource.com/c/crosvm/crosvm/+/506532
Kind regards,
Alexander
[1]
https://lore.kernel.org/virtio-dev/a9235fe7-7448-fa9f-ea52-fd27f4845bc4@opensynergy.com/
[2]
https://lore.kernel.org/virtio-dev/CAPBb6MXw5PebCXYBvXOP_b2j+t-1Y_rSJv7kWkLPsa3X+uM9gA@mail.gmail.com/
[3]
https://lore.kernel.org/virtio-dev/CAKxn3ec7nDy9B0nyHUg3GfXt_oktrC91kB3QWXM-sGh8ectE5A@mail.gmail.com/
[4] https://github.com/OpenSynergy/linux/tree/virtio-video
[5]
https://github.com/torvalds/linux/commit/b5c9ed70d1a94c59dad7b1ecfc928863c0fe6ac0
--
Alexander Gordeev
Senior Software Engineer
OpenSynergy GmbH
Rotherstr. 20, 10245 Berlin
www.opensynergy.com
---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org
prev parent reply other threads:[~2023-12-15 17:06 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-12-08 6:46 [virtio-dev] [FYI] virtio-media published Alexandre Courbot
2023-12-15 17:06 ` Alexander Gordeev [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=565caa81-c265-4986-80b5-5a0e0ebbfe6d@opensynergy.com \
--to=alexander.gordeev@opensynergy.com \
--cc=acourbot@chromium.org \
--cc=aesteve@redhat.com \
--cc=alex.bennee@linaro.org \
--cc=daniel.almeida@collabora.com \
--cc=eballetb@redhat.com \
--cc=egranata@google.com \
--cc=gustavo.padovan@collabora.com \
--cc=hverkuil@xs4all.nl \
--cc=keiichiw@chromium.org \
--cc=linux-media@vger.kernel.org \
--cc=nicolas.dufresne@collabora.com \
--cc=peter.griffin@linaro.org \
--cc=virtio-dev@lists.oasis-open.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).