From: Jonathan Cameron <Jonathan.Cameron@Huawei.com>
To: Richard Henderson <richard.henderson@linaro.org>
Cc: qemu-devel@nongnu.org, "Peter Maydell" <peter.maydell@linaro.org>,
"Gregory Price" <gregory.price@memverge.com>,
"Alex Bennée" <alex.bennee@linaro.org>,
"Sajjan Rao" <sajjanr@gmail.com>,
"Dimitrios Palyvos" <dimitrios.palyvos@zptcorp.com>,
"Paolo Bonzini" <pbonzini@redhat.com>,
"Eduardo Habkost" <eduardo@habkost.net>,
linuxarm@huawei.com, linux-cxl@vger.kernel.org
Subject: Re: [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL)
Date: Wed, 6 Mar 2024 15:04:04 +0000 [thread overview]
Message-ID: <20240306150404.000063cc@Huawei.com> (raw)
In-Reply-To: <1cf87044-8830-49da-92c8-406316260e51@linaro.org>
On Thu, 22 Feb 2024 09:07:03 -1000
Richard Henderson <richard.henderson@linaro.org> wrote:
> On 2/19/24 07:31, Jonathan Cameron wrote:
> > v2: Changes documented in patch 3.
> > - I have not addressed Richard's comment on recursive locks as that
> > seems to be a more general issue not specific to this patch set.
> >
> > CXL memory is interleaved at granularities as fine as 64 bytes.
> > To emulate this each read and write access undergoes address translation
> > similar to that used in physical hardware. This is done using
> > cfmws_ops for a memory region per CXL Fixed Memory Window (the PA address
> > range in the host that is interleaved across host bridges and beyond.
> > The OS programs interleaved decoders in the CXL Root Bridges, switch
> > upstream ports and the corresponding decoders CXL type 3 devices who
> > have to know the Host PA to Device PA mappings).
> >
> > Unfortunately this CXL memory may be used as normal memory and anything
> > that can end up in RAM can be placed within it. As Linux has become
> > more capable of handling this memory we've started to get quite a few
> > bug reports for the QEMU support. However terrible the performance is
> > people seem to like running actual software stacks on it :(
> >
> > This doesn't work for KVM - so for now CXL emulation remains TCG only.
> > (unless you are very careful on how it is used!) I plan to add some
> > safety guards at a later date to make it slightly harder for people
> > to shoot themselves in the foot + a more limited set of CXL functionality
> > that is safe (no interleaving!)
> >
> > Previously we had some issues with TCG reading instructions from CXL
> > memory but that is now all working. This time the issues are around
> > the Page Tables being in the CXL memory + DMA buffers being placed in it.
> >
> > The test setup I've been using is simple 2 way interleave via 2 root
> > ports below a single CXL root complex. After configuration in Linux
> > these are mapped to their own Numa Node and
> > numactl --membind=1 ls
> > followed by powering down the machine is sufficient to hit all the bugs
> > addressed in this series.
> >
> > Thanks to Gregory, Peter and Alex for their help figuring this lot
> > out. Note that I've included one patch from Peter without a SoB
> > because so far it has only be posted in the discussion thread.
> >
> > Whilst thread started back at:
> > https://lore.kernel.org/all/CAAg4PaqsGZvkDk_=PH+Oz-yeEUVcVsrumncAgegRKuxe_YoFhA@mail.gmail.com/
> > The QEMU part is from.
> > https://lore.kernel.org/all/20240201130438.00001384@Huawei.com/
> >
> >
> > Gregory Price (1):
> > target/i386: Enable page walking from MMIO memory
> >
> > Jonathan Cameron (1):
> > tcg: Avoid double lock if page tables happen to be in mmio memory.
> >
> > Peter Maydell (1):
> > accel/tcg: Set can_do_io at at start of lookup_tb_ptr helper
>
> Thanks. Queued patches 1 and 3 to tcg-next, and adjusted patch 3 to use BQL_LOCK_GUARD as
> suggested by Alex.
>
Thanks.
For patch 2, I assume this goes via an x86 specific path.
Paolo? Looks like most such patches go through you.
Jonathan
>
> r~
>
prev parent reply other threads:[~2024-03-06 15:04 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-02-19 17:31 [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL) Jonathan Cameron
2024-02-19 17:31 ` [PATCH v2 1/3] accel/tcg: Set can_do_io at at start of lookup_tb_ptr helper Jonathan Cameron
2024-02-19 17:31 ` [PATCH v2 2/3] target/i386: Enable page walking from MMIO memory Jonathan Cameron
2024-02-19 17:31 ` [PATCH v2 3/3] tcg: Avoid double lock if page tables happen to be in mmio memory Jonathan Cameron
2024-02-22 17:35 ` Peter Maydell
2024-02-22 19:07 ` [PATCH v2 0/3 qemu] tcg/i386: Page tables in MMIO memory fixes (CXL) Richard Henderson
2024-03-06 15:04 ` Jonathan Cameron [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240306150404.000063cc@Huawei.com \
--to=jonathan.cameron@huawei.com \
--cc=alex.bennee@linaro.org \
--cc=dimitrios.palyvos@zptcorp.com \
--cc=eduardo@habkost.net \
--cc=gregory.price@memverge.com \
--cc=linux-cxl@vger.kernel.org \
--cc=linuxarm@huawei.com \
--cc=pbonzini@redhat.com \
--cc=peter.maydell@linaro.org \
--cc=qemu-devel@nongnu.org \
--cc=richard.henderson@linaro.org \
--cc=sajjanr@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).