From: "T.J. Mercier" <tjmercier@google.com>
To: Christoph Hellwig <hch@lst.de>
Cc: Catalin Marinas <catalin.marinas@arm.com>,
Marek Szyprowski <m.szyprowski@samsung.com>,
Robin Murphy <robin.murphy@arm.com>,
isaacmanjarres@google.com, iommu@lists.linux.dev,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH] dma-direct: Set SG_DMA_SWIOTLB flag for dma-direct
Date: Thu, 9 May 2024 11:32:38 -0700 [thread overview]
Message-ID: <CABdmKX0KRA3NHiEJYsq5LqtwwEdM4LaONpyukd6zgk7hHzp3Cg@mail.gmail.com> (raw)
In-Reply-To: <20240509130659.GA12345@lst.de>
On Thu, May 9, 2024 at 6:07 AM Christoph Hellwig <hch@lst.de> wrote:
>
> On Thu, May 09, 2024 at 08:49:40AM +0100, Catalin Marinas wrote:
> > I see the swiotlb use as some internal detail of the DMA API
> > implementation that should not leak outside this framework.
>
> And that's what it is.
>
> > I think we should prevent bouncing if DMA_ATTR_SKIP_CPU_SYNC is passed.
> > However, this is not sufficient with a proper use of the DMA API since
> > the first dma_map_*() without this attribute can still do the bouncing.
> > IMHO what we need is a DMA_ATTR_NO_BOUNCE or DMA_ATTR_SHARED that will
> > be used on the first map and potentially on subsequent calls in
> > combination with DMA_ATTR_SKIP_CPU_SYNC (though we could use the latter
> > to imply "shared"). The downside is that mapping may fail if the
> > coherent mask is too narrow.
>
> We have two big problems here that kinda interact:
>
> 1) DMA_ATTR_SKIP_CPU_SYNC is just a horrible API. It exposes an
> implementation detail instead of dealing with use cases.
> The original one IIRC was to deal with networking receive
> buffers that are often only partially filled and the networking
> folks wanted to avoid the overhead for doing the cache operations
> for the rest. It kinda works for that but already gets iffy
> when swiotlb is involved. The other abuses of the flag just
> went downhill form there.
>
> 2) the model of dma mapping a single chunk of memory to multiple
> devices is not really well accounted for in the DMA API.
>
> So for two we need a memory allocator that can take the constraints
> of multiple devices into account, and probably a way to fail a
> dma-buf attach when the importer can't address the memory.
> We also then need to come up with a memory ownership / cache
> maintenance protocol that works for this use case.
Being able to fail the attach without necessarily performing any
mapping yet would be an improvement. However I think the original idea
was for dmabuf exporters to perform the constraint solving (if
possible) as attachments get added and then finally allocate however
is best when the buffer is first mapped. But as far as I know there
are no exporters that currently do this. Instead I think the problem
is currently being avoided by using custom exporters for particular
sets of usecases that are known to work on a given system. This
swiotlb + uncached example is one reason we'd want to fail the
constraint solving. The DMA API knows about the swiotlb part but not
really about the uncached part.
prev parent reply other threads:[~2024-05-09 18:32 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-03 18:37 [PATCH] dma-direct: Set SG_DMA_SWIOTLB flag for dma-direct T.J. Mercier
2024-05-04 8:53 ` Petr Tesařík
2024-05-09 13:28 ` Robin Murphy
2024-05-06 5:29 ` Christoph Hellwig
2024-05-06 16:00 ` T.J. Mercier
2024-05-06 16:02 ` Christoph Hellwig
2024-05-06 16:10 ` T.J. Mercier
2024-05-06 16:19 ` Christoph Hellwig
2024-05-06 16:39 ` T.J. Mercier
2024-05-07 5:43 ` Christoph Hellwig
2024-05-07 20:07 ` T.J. Mercier
2024-05-08 11:33 ` Christoph Hellwig
2024-05-09 12:54 ` Robin Murphy
2024-05-09 18:26 ` T.J. Mercier
2024-05-08 17:19 ` Catalin Marinas
2024-05-08 20:14 ` T.J. Mercier
2024-05-09 7:49 ` Catalin Marinas
2024-05-09 13:06 ` Christoph Hellwig
2024-05-09 18:32 ` T.J. Mercier [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CABdmKX0KRA3NHiEJYsq5LqtwwEdM4LaONpyukd6zgk7hHzp3Cg@mail.gmail.com \
--to=tjmercier@google.com \
--cc=catalin.marinas@arm.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux.dev \
--cc=isaacmanjarres@google.com \
--cc=linux-kernel@vger.kernel.org \
--cc=m.szyprowski@samsung.com \
--cc=robin.murphy@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).