From: Yuquan Wang <wangyuquan1236@phytium.com.cn>
To: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: qemu-devel@nongnu.org, linux-cxl@vger.kernel.org
Subject: Re: CXL numa error on arm64 qemu virt machine
Date: Thu, 9 May 2024 16:35:34 +0800 [thread overview]
Message-ID: <ZjyK1tI0TtbAKhmh@phytium.com.cn> (raw)
In-Reply-To: <20240508130252.00006367@Huawei.com>
On Wed, May 08, 2024 at 01:02:52PM +0100, Jonathan Cameron wrote:
>
> > [ 0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x40000000-0xbfffffff]
> > [ 0.000000] ACPI: SRAT: Node 1 PXM 1 [mem 0xc0000000-0x13fffffff]
> > [ 0.000000] ACPI: Unknown target node for memory at 0x10000000000, assuming node 0
> > [ 0.000000] NUMA: Warning: invalid memblk node 16 [mem 0x0000000004000000-0x0000000007ffffff]
> > [ 0.000000] NUMA: Faking a node at [mem 0x0000000004000000-0x000000013fffffff]
> > [ 0.000000] NUMA: NODE_DATA [mem 0x13f7f89c0-0x13f7fafff]
> >
> > Previous discussion: https://lore.kernel.org/linux-cxl/20231011150620.0000212a@Huawei.com/
> >
> > root@debian-bullseye-arm64:~# cxl create-region -d decoder0.0 -t ram
> > [ 68.653873] cxl region0: Bypassing cpu_cache_invalidate_memregion() for testing!
> > [ 68.660568] Unknown target node for memory at 0x10000000000, assuming node 0
>
> You need a load of kernel changes for NUMA nodes to work correctly with
> CXL memory on arm64 platforms. I have some working code but need to tidy
> up a few corners that came up in an internal review earlier this week.
>
> I have some travel coming up so may take a week or so to get those out.
>
> Curiously that invalid memblk has nothing to do with the CXL fixed memory window
> Could you check if that is happening for you without the CXL patches?
>
Thanks.
I have checked it, the problem is caused by my bios firmware file. I
change the bios file and the numa topology can works.
BTW, if it is convenient, could you help to post the link of the patches of CXL
memory NUMA nodes on arm64 platforms?
>
> Whilst it doesn't work yet (because of missing kernel support)
> you'll need something that looks more like the generic ports test added in
> https://gitlab.com/jic23/qemu/-/commit/6589c527920ba22fe0923b60b58d33a8e9fd371e
>
> Most importantly
> -numa node,nodeid=2 -object acpi-generic-port,id=gp0,pci-bus-cxl.1,node=2
> + the bits setting distances etc. Note CXL memory does not provide SLIT like
> data at the moment, so the test above won't help you identify if it is correctly
> set up. That's a gap in general in the kernel support. Whilst we'd love
> it if everyone moved to hmat derived information we may need to provide
> some fallback.
>
> Jonathan
>
Many thanks
Yuquan
next prev parent reply other threads:[~2024-05-09 8:37 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-08 8:00 CXL numa error on arm64 qemu virt machine Yuquan Wang
2024-05-08 12:02 ` Jonathan Cameron via
2024-05-09 8:35 ` Yuquan Wang [this message]
2024-05-10 17:16 ` Jonathan Cameron via
2024-05-17 10:07 ` Yuquan Wang
2024-05-17 10:14 ` Jonathan Cameron via
2024-05-17 18:03 ` Jonathan Cameron via
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZjyK1tI0TtbAKhmh@phytium.com.cn \
--to=wangyuquan1236@phytium.com.cn \
--cc=20240508130252.00006367@huawei.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=linux-cxl@vger.kernel.org \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).