From: Gregory Price <gregory.price@memverge.com>
To: Dan Williams <dan.j.williams@intel.com>
Cc: linux-cxl@vger.kernel.org, Dave Jiang <dave.jiang@intel.com>
Subject: Re: [BUG] DAX access of Memory Expander on RCH topology fires BUG on page_table_check
Date: Wed, 19 Apr 2023 20:58:10 -0400 [thread overview]
Message-ID: <ZECOIrOqXZgSOLsb@memverge.com> (raw)
In-Reply-To: <643e3c0f22afd_556e2941c@dwillia2-mobl3.amr.corp.intel.com.notmuch>
On Mon, Apr 17, 2023 at 11:43:27PM -0700, Dan Williams wrote:
> Gregory Price wrote:
> > Why NUMA-mode works under these conditions without crashing the system
> > is escaping me at the moment,
>
> Why would it crash? That range is valid within
> 0x1050000000-0x304fffffff.
>
Basically I was expecting a page-fault in NUMA to produce the same
effects as a fault fault in DAX, clearly this is not the case and either
the switch from numa to dax is causing
> > given that the page faulting system goes
> > through the same driver. But my guess is that pfn-to-page mappings are
> > off in some way when placed in devdax mode, whereas they're correct
> > under numa mode.
>
> pfn-to-page is pretty simple, its the pfn to page_ext that's concerning
> for CONFIG_PAGE_TABLE_CHECK.
>
Testing CONFIG_PAGE_TABLE_CHECK=n now, will report back when done.
> > Note that the above code chops off the first 768MB of the dax region and
> > the last 1.25GB of the dax region.
>
> Yes, if the core-mm picks 2GB for the block size (which it does for
> systems with more the 64GB of memory, then it will align hot-added
> ranges.
>
> > The CFWM is required to be 256MB aligned, but this code will force
> > anything mapped into that area to be 2GB aligned. I don't think it's
> > safe to safe the BIOS is wrong.
>
> The *minimum* alignment of the CFMWS window is 256M, but if they don't
> want to waste memory on Linux they had better make it 2GB aligned.
>
> BIOS looks ok here.
>
FWIW i have a QEMU instance with 64GB that puts CXL devices on 256MB
alignment as well, so QEMU instances over a certain amount of DRAM
produce the same effect as hardware - lost memory.
~Gregory
next prev parent reply other threads:[~2023-04-20 0:58 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-12 18:43 [BUG] DAX access of Memory Expander on RCH topology fires BUG on page_table_check Gregory Price
2023-04-13 11:39 ` Gregory Price
2023-04-18 6:43 ` Dan Williams
2023-04-20 0:58 ` Gregory Price [this message]
2023-04-18 6:35 ` Dan Williams
2023-04-20 1:29 ` Gregory Price
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=ZECOIrOqXZgSOLsb@memverge.com \
--to=gregory.price@memverge.com \
--cc=dan.j.williams@intel.com \
--cc=dave.jiang@intel.com \
--cc=linux-cxl@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.