From: Magnus Karlsson <magnus.karlsson@gmail.com>
To: "Július Milan" <julius.milan.22@gmail.com>
Cc: xdp-newbies@vger.kernel.org
Subject: Re: possibility to fill completion queue from user space?
Date: Mon, 3 Jul 2023 12:14:45 +0200 [thread overview]
Message-ID: <CAJ8uoz0mV2pAvCyJVH5cvckQDovC-q+sUCLMM5siKOrfT+m4Yw@mail.gmail.com> (raw)
In-Reply-To: <CANaGk7tkiDwmD_18cE6TioRs0cO5F5bX4n1msU-YZnVopAO0cg@mail.gmail.com>
On Mon, 3 Jul 2023 at 11:59, Július Milan <julius.milan.22@gmail.com> wrote:
>
> Thanks a lot Magnus
>
> If I understand correctly, you mean creating some storage of free umem
> addresses?
> Something as:
> https://github.com/xdp-project/xdp-tutorial/blob/master/advanced03-AF_XDP/af_xdp_user.c#L147
> If so I did it and it works well, I just have to keep them separated per thread.
>
> Regarding the new peek you mentioned, do you think such patch would be
> meaningful even for
> merging in libxdp? That would be a motivation for me to dive deep into it.
> And would it be thread safe as I suppose?
It would not be thread safe and I would not spend time trying to merge
it to libxdp since you are the only one in 5-years that have requested
something like this :-). You have a solution that works for you. Just
go on and focus on what you really would like to accomplish.
> One more question: can I use the poll() also for the Completion Queue?
> It might be useful for me.
This is not supported, but I can see the use of it. Wakeup when there
is a new entry in the completion queue I guess. This has been
requested by someone before, but not been implemented so far.
> Best Regards
> Julius
>
> On Thu, Jun 29, 2023 at 8:32 AM Magnus Karlsson
> <magnus.karlsson@gmail.com> wrote:
> >
> > On Wed, 28 Jun 2023 at 20:26, Július Milan <julius.milan.22@gmail.com> wrote:
> > >
> > > Hi all
> > >
> > > I am writing an AF_XDP based user space application.
> > > However in my use case, packets payload get fragmented while
> > > processing, basically new packets are constructed inside and sent
> > > further.
> > > I probably cannot avoid mempcy anyway.
> > >
> > > So I plan to solve it simply - one umem per port, no locking, no
> > > keeping track of umem frames presence (kernel / user space) . Just
> > > usage of the rings, one half of the frames to circulate between the RX
> > > <-> fill queue, the other half TX <-> completion queue.
> > >
> > > Is it actually possible to initialize the rings in such a way that at
> > > the very beginning I would fill the completion queue by some frames?
> > > This is to avoid multithreaded access to the free frames without
> > > locking (initial TX would take a look for free frames inside the
> > > completion queue).
> >
> > You could fill the completion ring with entries from user space at
> > initialization time. As long as you always pick the first entry in the
> > completion ring before putting it on the Tx ring, the kernel would not
> > overwrite your entries. One complication is that you would have to
> > construct a new peek() routine for the completion queue, as the normal
> > one would indicate no entries found even though you have written
> > entries in it.
> >
> > Maybe an easier idea is to just have some code like this:
> >
> > /* Total N entries in Tx and completion ring.
> > * allocated initialized to 0 somewhere */
> > if (allocated < N) {
> > allocated++;
> > return N;
> > }
> > return next entry on completion queue;
> >
> > Returns the buffer number in the umem and assumes your Tx buffers are
> > first in the umem. Though this scheme would introduce an if statement
> > in the path, it would be easier to start with. You need an address not
> > a number, and you likely also need some indication of "no buffer
> > available". Tried to keep it simple.
> >
> >
> > > If it's a bad idea, what else would you suggest?
> > >
> > > Thank you
> > > Julius
prev parent reply other threads:[~2023-07-03 10:15 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-06-28 18:06 possibility to fill completion queue from user space? Július Milan
2023-06-29 6:31 ` Magnus Karlsson
2023-07-03 9:59 ` Július Milan
2023-07-03 10:14 ` Magnus Karlsson [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAJ8uoz0mV2pAvCyJVH5cvckQDovC-q+sUCLMM5siKOrfT+m4Yw@mail.gmail.com \
--to=magnus.karlsson@gmail.com \
--cc=julius.milan.22@gmail.com \
--cc=xdp-newbies@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).