From: Robin Murphy <robin.murphy@arm.com>
To: Konrad Dybcio <konrad.dybcio@linaro.org>,
Dmitry Baryshkov <dmitry.baryshkov@linaro.org>,
"open list:DRM DRIVER FOR MSM ADRENO GPU"
<linux-arm-msm@vger.kernel.org>,
Bjorn Andersson <andersson@kernel.org>
Cc: Joerg Roedel <joro@8bytes.org>, Christoph Hellwig <hch@lst.de>,
Vineet Gupta <vgupta@kernel.org>,
Russell King <linux@armlinux.org.uk>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Huacai Chen <chenhuacai@kernel.org>,
WANG Xuerui <kernel@xen0n.name>,
Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
Paul Walmsley <paul.walmsley@sifive.com>,
Palmer Dabbelt <palmer@dabbelt.com>,
Albert Ou <aou@eecs.berkeley.edu>,
Lorenzo Pieralisi <lpieralisi@kernel.org>,
Hanjun Guo <guohanjun@huawei.com>,
Sudeep Holla <sudeep.holla@arm.com>,
"K. Y. Srinivasan" <kys@microsoft.com>,
Haiyang Zhang <haiyangz@microsoft.com>,
Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
David Woodhouse <dwmw2@infradead.org>,
Lu Baolu <baolu.lu@linux.intel.com>,
Niklas Schnelle <schnelle@linux.ibm.com>,
Matthew Rosato <mjrosato@linux.ibm.com>,
Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
Rob Herring <robh+dt@kernel.org>,
Frank Rowand <frowand.list@gmail.com>,
Marek Szyprowski <m.szyprowski@samsung.com>,
Jason Gunthorpe <jgg@ziepe.ca>,
linux-kernel@vger.kernel.org,
linux-arm-kernel@lists.infradead.org, linux-acpi@vger.kernel.org,
iommu@lists.linux.dev, devicetree@vger.kernel.org,
Jason Gunthorpe <jgg@nvidia.com>
Subject: Re: [PATCH v4 6/7] iommu/dma: Centralise iommu_setup_dma_ops()
Date: Tue, 30 Apr 2024 13:33:20 +0100 [thread overview]
Message-ID: <1e691960-bc52-4e5c-89d7-ecd3f7dc9f56@arm.com> (raw)
In-Reply-To: <ebc8813c-74eb-49d1-b8d0-a6f1821f711a@linaro.org>
On 30/04/2024 1:23 pm, Konrad Dybcio wrote:
> On 29.04.2024 11:26 PM, Dmitry Baryshkov wrote:
>> On Mon, 29 Apr 2024 at 19:31, Dmitry Baryshkov
>> <dmitry.baryshkov@linaro.org> wrote:
>>>
>>> On Fri, Apr 19, 2024 at 05:54:45PM +0100, Robin Murphy wrote:
>>>> It's somewhat hard to see, but arm64's arch_setup_dma_ops() should only
>>>> ever call iommu_setup_dma_ops() after a successful iommu_probe_device(),
>>>> which means there should be no harm in achieving the same order of
>>>> operations by running it off the back of iommu_probe_device() itself.
>>>> This then puts it in line with the x86 and s390 .probe_finalize bodges,
>>>> letting us pull it all into the main flow properly. As a bonus this lets
>>>> us fold in and de-scope the PCI workaround setup as well.
>>>>
>>>> At this point we can also then pull the call up inside the group mutex,
>>>> and avoid having to think about whether iommu_group_store_type() could
>>>> theoretically race and free the domain if iommu_setup_dma_ops() ran just
>>>> *before* iommu_device_use_default_domain() claims it... Furthermore we
>>>> replace one .probe_finalize call completely, since the only remaining
>>>> implementations are now one which only needs to run once for the initial
>>>> boot-time probe, and two which themselves render that path unreachable.
>>>>
>>>> This leaves us a big step closer to realistically being able to unpick
>>>> the variety of different things that iommu_setup_dma_ops() has been
>>>> muddling together, and further streamline iommu-dma into core API flows
>>>> in future.
>>>>
>>>> Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com> # For Intel IOMMU
>>>> Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>
>>>> Tested-by: Hanjun Guo <guohanjun@huawei.com>
>>>> Signed-off-by: Robin Murphy <robin.murphy@arm.com>
>>>> ---
>>>> v2: Shuffle around to make sure the iommu_group_do_probe_finalize() case
>>>> is covered as well, with bonus side-effects as above.
>>>> v3: *Really* do that, remembering the other two probe_finalize sites too.
>>>> ---
>>>> arch/arm64/mm/dma-mapping.c | 2 --
>>>> drivers/iommu/amd/iommu.c | 8 --------
>>>> drivers/iommu/dma-iommu.c | 18 ++++++------------
>>>> drivers/iommu/dma-iommu.h | 14 ++++++--------
>>>> drivers/iommu/intel/iommu.c | 7 -------
>>>> drivers/iommu/iommu.c | 20 +++++++-------------
>>>> drivers/iommu/s390-iommu.c | 6 ------
>>>> drivers/iommu/virtio-iommu.c | 10 ----------
>>>> include/linux/iommu.h | 7 -------
>>>> 9 files changed, 19 insertions(+), 73 deletions(-)
>>>
>>> This patch breaks UFS on Qualcomm SC8180X Primus platform:
>>>
>>>
>>> [ 3.846856] arm-smmu 15000000.iommu: Unhandled context fault: fsr=0x402, iova=0x1032db3e0, fsynr=0x130000, cbfrsynra=0x300, cb=4
>>> [ 3.846880] ufshcd-qcom 1d84000.ufshc: ufshcd_check_errors: saved_err 0x20000 saved_uic_err 0x0
>>> [ 3.846929] host_regs: 00000000: 1587031f 00000000 00000300 00000000
>>> [ 3.846935] host_regs: 00000010: 01000000 00010217 00000000 00000000
>>> [ 3.846941] host_regs: 00000020: 00000000 00070ef5 00000000 00000000
>>> [ 3.846946] host_regs: 00000030: 0000000f 00000001 00000000 00000000
>>> [ 3.846951] host_regs: 00000040: 00000000 00000000 00000000 00000000
>>> [ 3.846956] host_regs: 00000050: 032db000 00000001 00000000 00000000
>>> [ 3.846962] host_regs: 00000060: 00000000 80000000 00000000 00000000
>>> [ 3.846967] host_regs: 00000070: 032dd000 00000001 00000000 00000000
>>> [ 3.846972] host_regs: 00000080: 00000000 00000000 00000000 00000000
>>> [ 3.846977] host_regs: 00000090: 00000016 00000000 00000000 0000000c
>>> [ 3.847074] ufshcd-qcom 1d84000.ufshc: ufshcd_err_handler started; HBA state eh_fatal; powered 1; shutting down 0; saved_err = 131072; saved_uic_err = 0; force_reset = 0
>>> [ 4.406550] ufshcd-qcom 1d84000.ufshc: ufshcd_verify_dev_init: NOP OUT failed -11
>>> [ 4.417953] ufshcd-qcom 1d84000.ufshc: ufshcd_async_scan failed: -11
>>
>> Just to confirm: reverting f091e93306e0 ("dma-mapping: Simplify
>> arch_setup_dma_ops()") and b67483b3c44e ("iommu/dma: Centralise
>> iommu_setup_dma_ops()" fixes the issue for me. Please ping me if you'd
>> like me to test a fix.
>
> This also triggers a different issue (that also comes down to "ufs bad") on
> another QC platform (SM8550):
>
> [ 4.282098] scsi host0: ufshcd
> [ 4.315970] ufshcd-qcom 1d84000.ufs: ufshcd_check_errors: saved_err 0x20000 saved_uic_err 0x0
> [ 4.330155] host_regs: 00000000: 3587031f 00000000 00000400 00000000
> [ 4.343955] host_regs: 00000010: 01000000 00010217 00000000 00000000
> [ 4.356027] host_regs: 00000020: 00000000 00070ef5 00000000 00000000
> [ 4.370136] host_regs: 00000030: 0000000f 00000003 00000000 00000000
> [ 4.376662] host_regs: 00000040: 00000000 00000000 00000000 00000000
> [ 4.383192] host_regs: 00000050: 85109000 00000008 00000000 00000000
> [ 4.389719] host_regs: 00000060: 00000000 80000000 00000000 00000000
> [ 4.396245] host_regs: 00000070: 8510a000 00000008 00000000 00000000
> [ 4.402773] host_regs: 00000080: 00000000 00000000 00000000 00000000
> [ 4.409298] host_regs: 00000090: 00000016 00000000 00000000 0000000c
> [ 4.415900] arm-smmu 15000000.iommu: Unhandled context fault: fsr=0x402, iova=0x8851093e0, fsynr=0x3b0001, cbfrsynra=0x60, cb=2
> [ 4.416135] ufshcd-qcom 1d84000.ufs: ufshcd_err_handler started; HBA state eh_fatal; powered 1; shutting down 0; saved_err = 131072; saved_uic_err = 0; force_reset = 0
> [ 4.951750] ufshcd-qcom 1d84000.ufs: ufshcd_verify_dev_init: NOP OUT failed -11
> [ 4.960644] ufshcd-qcom 1d84000.ufs: ufshcd_async_scan failed: -11
>
> Reverting the commits Dmitry mentioned also fixes this.
Yeah, It'll be the same thing - doesn't really matter exactly *how* the
UFS goes wrong due to the SMMU blocking it, the issue is that the SMMU
is erroneously blocking it in the first place due to a DMA ops mixup.
Fix is now here:
https://lore.kernel.org/linux-iommu/d4cc20cbb0c45175e98dd76bf187e2ad6421296d.1714472573.git.robin.murphy@arm.com/
Thanks,
Robin.
prev parent reply other threads:[~2024-04-30 12:33 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <cover.1713523152.git.robin.murphy@arm.com>
[not found] ` <bebea331c1d688b34d9862eefd5ede47503961b8.1713523152.git.robin.murphy@arm.com>
[not found] ` <Zi_LV28TR-P-PzXi@eriador.lumag.spb.ru>
2024-04-29 21:26 ` [PATCH v4 6/7] iommu/dma: Centralise iommu_setup_dma_ops() Dmitry Baryshkov
2024-04-30 12:23 ` Konrad Dybcio
2024-04-30 12:33 ` Robin Murphy [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1e691960-bc52-4e5c-89d7-ecd3f7dc9f56@arm.com \
--to=robin.murphy@arm.com \
--cc=andersson@kernel.org \
--cc=aou@eecs.berkeley.edu \
--cc=baolu.lu@linux.intel.com \
--cc=catalin.marinas@arm.com \
--cc=chenhuacai@kernel.org \
--cc=decui@microsoft.com \
--cc=devicetree@vger.kernel.org \
--cc=dmitry.baryshkov@linaro.org \
--cc=dwmw2@infradead.org \
--cc=frowand.list@gmail.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=guohanjun@huawei.com \
--cc=haiyangz@microsoft.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux.dev \
--cc=jean-philippe@linaro.org \
--cc=jgg@nvidia.com \
--cc=jgg@ziepe.ca \
--cc=joro@8bytes.org \
--cc=kernel@xen0n.name \
--cc=konrad.dybcio@linaro.org \
--cc=kys@microsoft.com \
--cc=linux-acpi@vger.kernel.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-arm-msm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=lpieralisi@kernel.org \
--cc=m.szyprowski@samsung.com \
--cc=mjrosato@linux.ibm.com \
--cc=palmer@dabbelt.com \
--cc=paul.walmsley@sifive.com \
--cc=robh+dt@kernel.org \
--cc=schnelle@linux.ibm.com \
--cc=sudeep.holla@arm.com \
--cc=suravee.suthikulpanit@amd.com \
--cc=tsbogend@alpha.franken.de \
--cc=vgupta@kernel.org \
--cc=wei.liu@kernel.org \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).