From: Niklas Cassel <cassel@kernel.org>
To: Lennert Buytenhek <kernel@wantstofly.org>
Cc: Damien Le Moal <dlemoal@kernel.org>,
linux-ide@vger.kernel.org, linux-kernel@vger.kernel.org,
Robin Murphy <robin.murphy@arm.com>,
John Garry <john.g.garry@oracle.com>,
Joerg Roedel <jroedel@suse.de>
Subject: Re: [PATCH v2] ahci: add 43-bit DMA address quirk for ASMedia ASM1061 controllers
Date: Thu, 25 Jan 2024 17:19:07 +0100 [thread overview]
Message-ID: <ZbKJ++oBcTptpvlB@x1-carbon> (raw)
In-Reply-To: <ZbJ4Yfexzea2mQw8@wantstofly.org>
On Thu, Jan 25, 2024 at 05:04:01PM +0200, Lennert Buytenhek wrote:
> With one of the on-board ASM1061 AHCI controllers (1b21:0612) on an
> ASUSTeK Pro WS WRX80E-SAGE SE WIFI mainboard, a controller hang was
> observed that was immediately preceded by the following kernel
> messages:
>
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: Using 64-bit DMA addresses
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00000 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00300 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00380 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00400 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00680 flags=0x0000]
> [Thu Jan 4 23:12:54 2024] ahci 0000:28:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0035 address=0x7fffff00700 flags=0x0000]
>
> The first message is produced by code in drivers/iommu/dma-iommu.c
> which is accompanied by the following comment that seems to apply:
>
> /*
> * Try to use all the 32-bit PCI addresses first. The original SAC vs.
> * DAC reasoning loses relevance with PCIe, but enough hardware and
> * firmware bugs are still lurking out there that it's safest not to
> * venture into the 64-bit space until necessary.
> *
> * If your device goes wrong after seeing the notice then likely either
> * its driver is not setting DMA masks accurately, the hardware has
> * some inherent bug in handling >32-bit addresses, or not all the
> * expected address bits are wired up between the device and the IOMMU.
> */
>
> Asking the ASM1061 on a discrete PCIe card to DMA from I/O virtual
> address 0xffffffff00000000 produces the following I/O page faults:
>
> [Tue Jan 23 21:31:55 2024] vfio-pci 0000:07:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0021 address=0x7ff00000000 flags=0x0010]
> [Tue Jan 23 21:31:55 2024] vfio-pci 0000:07:00.0: AMD-Vi: Event logged [IO_PAGE_FAULT domain=0x0021 address=0x7ff00000500 flags=0x0010]
>
> Note that the upper 21 bits of the logged DMA address are zero. (When
> asking a different PCIe device in the same PCIe slot to DMA to the
> same I/O virtual address, we do see all the upper 32 bits of the DMA
> address as 1, so this is not an issue with the chipset or IOMMU
> configuration on the test system.)
>
> Also, hacking libahci to always set the upper 21 bits of all DMA
> addresses to 1 produces no discernible effect on the behavior of the
> ASM1061, and mkfs/mount/scrub/etc work as without this hack.
>
> This all strongly suggests that the ASM1061 has a 43 bit DMA address
> limit, and this commit therefore adds a quirk to deal with this limit.
>
> This issue probably applies to (some of) the other supported ASMedia
> parts as well, but we limit it to the PCI IDs known to refer to
> ASM1061 parts, as that's the only part we know for sure to be affected
> by this issue at this point.
>
> Link: https://lore.kernel.org/linux-ide/ZaZ2PIpEId-rl6jv@wantstofly.org/
> Signed-off-by: Lennert Buytenhek <kernel@wantstofly.org>
> ---
Applied:
https://git.kernel.org/pub/scm/linux/kernel/git/libata/linux.git/log/?h=for-6.8-fixes
prev parent reply other threads:[~2024-01-25 16:19 UTC|newest]
Thread overview: 2+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-25 15:04 [PATCH v2] ahci: add 43-bit DMA address quirk for ASMedia ASM1061 controllers Lennert Buytenhek
2024-01-25 16:19 ` Niklas Cassel [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZbKJ++oBcTptpvlB@x1-carbon \
--to=cassel@kernel.org \
--cc=dlemoal@kernel.org \
--cc=john.g.garry@oracle.com \
--cc=jroedel@suse.de \
--cc=kernel@wantstofly.org \
--cc=linux-ide@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=robin.murphy@arm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).