From: wuzongyong <wuzongyo@mail.ustc.edu.cn>
To: linux-kernel@vger.kernel.org, thomas.lendacky@amd.com,
kvm@vger.kernel.org, x86@kernel.org
Cc: linux-coco@lists.linux.dev
Subject: [Question] int3 instruction generates a #UD in SEV VM
Date: Sat, 29 Jul 2023 11:15:04 +0800 [thread overview]
Message-ID: <8eb933fd-2cf3-d7a9-32fe-2a1d82eac42a@mail.ustc.edu.cn> (raw)
Hi,
I am writing a firmware in Rust to support SEV based on project td-shim[1].
But when I create a SEV VM (just SEV, no SEV-ES and no SEV-SNP) with the firmware,
the linux kernel crashed because the int3 instruction in int3_selftest() cause a
#UD.
The stack is as follows:
[ 0.141804] invalid opcode: 0000 [#1] PREEMPT SMP^M
[ 0.141804] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.3.0+ #37^M
[ 0.141804] RIP: 0010:int3_selftest_ip+0x0/0x2a^M
[ 0.141804] Code: eb bc 66 90 0f 1f 44 00 00 48 83 ec 08 48 c7 c7 90 0d 78 83
c7 44 24 04 00 00 00 00 e8 23 fe ac fd 85 c0 75 22 48 8d 7c 24 04 <cc>
90 90 90 90 83 7c 24 04 01 75 13 48 c7 c7 90 0d 78 83 e8 42 fc^M
[ 0.141804] RSP: 0000:ffffffff82803f18 EFLAGS: 00010246^M
[ 0.141804] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000007ffffffe^M
[ 0.141804] RDX: ffffffff82fd4938 RSI: 0000000000000296 RDI: ffffffff82803f1c^M
[ 0.141804] RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000fffeffff^M
[ 0.141804] R10: ffffffff82803e08 R11: ffffffff82f615a8 R12: 00000000ff062350^M
[ 0.141804] R13: 000000001fddc20a R14: 000000000090122c R15: 0000000002000000^M
[ 0.141804] FS: 0000000000000000(0000) GS:ffff88801f200000(0000) knlGS:0000000000000000^M
[ 0.141804] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
[ 0.141804] CR2: ffff888004c00000 CR3: 000800000281f000 CR4: 00000000003506f0^M
[ 0.141804] Call Trace:^M
[ 0.141804] <TASK>^M
[ 0.141804] alternative_instructions+0xe/0x100^M
[ 0.141804] check_bugs+0xa7/0x110^M
[ 0.141804] start_kernel+0x320/0x430^M
[ 0.141804] secondary_startup_64_no_verify+0xd3/0xdb^M
[ 0.141804] </TASK>^M
[ 0.141804] Modules linked in:^M
[ 0.141804] ---[ end trace 0000000000000000 ]--
Then I tried to figure out the problem and do some test with qemu & OVMF in SEV.
But the behaviour is also weird when I create SEV VM with qemu & OVMF.
I found the int3 instruction always generated a #UD if I put a int3 instruction before
gen_pool_create() in mce_gen_pool_create(). But if I put the int3 instruction after the
gen_pool_create() in mce_gen_pool_create(), the int3 instruction generated a #BP rightly.
// linux/arch/x86/kernel/cpu/mce/genpool.c
static int mce_gen_pool_create(void)
{
struct gen_pool *tmpp;
int ret = -ENOMEM;
asm volatile ("int3\n\t"); // generated a # UD
tmpp = gen_pool_create(ilog2(sizeof(struct mce_evt_list)), -1);
asm volatile ("int3\n\t"); // generated a #BP
...
}
The stack is as follows when I put the int3 instruction before gen_pool_create().
[ 0.094846] invalid opcode: 0000 [#1] PREEMPT SMP^M
[ 0.094994] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.3.0+ #101^M
[ 0.094994] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 2/2/2022^M
[ 0.094994] RIP: 0010:mcheck_cpu_init+0x4e/0x150^M
[ 0.094994] Code: 84 c0 0f 89 97 00 00 00 48 8b 45 28 f6 c4 40 0f 84 8a 00 00 00 e8 c2
e6 ff ff 48 89 ef e8 8a e2 ff ff 85 c0 0f 88 94 00 00 00 <cc> e8 dc 05 00 00 85 c0 75 76
80 0d a1 90 0a 02 20 0f b6 45 01 3c^M
[ 0.094994] RSP: 0000:ffffffff92803ef8 EFLAGS: 00010246^M
[ 0.094994] RAX: 0000000000000000 RBX: 0000000000000058 RCX: 00000000ffffffff^M
[ 0.094994] RDX: 0000000000000002 RSI: 00000000000000ff RDI: ffffffff930ed860^M
[ 0.094994] RBP: ffffffff930ed860 R08: 0000000000000000 R09: 0000000000000000^M
[ 0.094994] R10: 0000000000000000 R11: 0000000000000254 R12: 0000000000000207^M
[ 0.094994] R13: 000000001f9ec018 R14: 000000001fe85928 R15: 0000000000000001^M
[ 0.094994] FS: 0000000000000000(0000) GS:ffff8ae0dca00000(0000) knlGS:0000000000000000^M
[ 0.094994] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
[ 0.094994] CR2: ffff8ae0dac01000 CR3: 000800001881f000 CR4: 00000000003506f0^M
[ 0.094994] Call Trace:^M
[ 0.094994] <TASK>^M
[ 0.094994] identify_cpu+0x2cb/0x500^M
[ 0.094994] identify_boot_cpu+0x10/0xb0^M
[ 0.094994] check_bugs+0xf/0x110^M
[ 0.094994] start_kernel+0x320/0x430^M
[ 0.094994] secondary_startup_64_no_verify+0xd3/0xdb^M
[ 0.094994] </TASK>^M
[ 0.094994] Modules linked in:^M
[ 0.094995] ---[ end trace 0000000000000000 ]---^
The stack is as follows when I put the int3 instruction after gen_pool_create().
[ 0.095585] int3: 0000 [#1] PREEMPT SMP^M
[ 0.095590] CPU: 0 PID: 0 Comm: swapper/0 Not tainted 6.3.0+ #101^M
[ 0.095593] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 2/2/2022^M
[ 0.095594] RIP: 0010:mcheck_cpu_init+0x4f/0x150^M
[ 0.095597] Code: c0 0f 89 97 00 00 00 48 8b 45 28 f6 c4 40 0f 84 8a 00 00 00 e8 c2 e6
ff ff 48 89 ef e8 8a e2 ff ff 85 c0 0f 88 94 00 00 00 cc <e8> dc 05 00 00 85 c0 75 76 80
0d a1 90 0a 02 20 0f b6 45 01 3c 02^M
[ 0.095598] RSP: 0000:ffffffff86803ef8 EFLAGS: 00000246^M
[ 0.095599] RAX: 0000000000000000 RBX: 0000000000000058 RCX: 00000000ffffffff^M
[ 0.095600] RDX: 0000000000000002 RSI: 00000000000000ff RDI: ffffffff870ed860^M
[ 0.095601] RBP: ffffffff870ed860 R08: 0000000000000000 R09: 0000000000000000^M
[ 0.095601] R10: 0000000000000000 R11: 0000000000000254 R12: 0000000000000207^M
[ 0.095602] R13: 000000001f9ec018 R14: 000000001fe85928 R15: 0000000000000001^M
[ 0.095604] FS: 0000000000000000(0000) GS:ffff901e5ca00000(0000) knlGS:0000000000000000^M
[ 0.095605] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033^M
[ 0.095606] CR2: ffff901e5ac01000 CR3: 000800001881f000 CR4: 00000000003506f0^M
[ 0.095606] Call Trace:^M
[ 0.095611] <TASK>^M
[ 0.095612] identify_cpu+0x2cb/0x500^M
[ 0.095615] identify_boot_cpu+0x10/0xb0^M
[ 0.095618] check_bugs+0xf/0x110^M
[ 0.095620] start_kernel+0x320/0x430^M
[ 0.095622] secondary_startup_64_no_verify+0xd3/0xdb^M
[ 0.095625] </TASK>^M
[ 0.095625] Modules linked in:^M
[ 0.096577] ---[ end trace 0000000000000000 ]---^
BTW, if a create a normal VM without SEV by qemu & OVMF, the int3 instruction always generates a
#BP.
So I am confused now about the behaviour of int3 instruction, could anyone help to explain the behaviour?
Any suggestion is appreciated!
[1] https://github.com/confidential-containers/td-shim
Thanks,
Wu Zongyo
next reply other threads:[~2023-07-29 3:20 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-07-29 3:15 wuzongyong [this message]
2023-07-31 14:30 ` [Question] int3 instruction generates a #UD in SEV VM Sean Christopherson
2023-07-31 15:03 ` Tom Lendacky
2023-07-31 15:45 ` wuzongyong
2023-08-02 11:56 ` Wu Zongyo
2023-08-02 14:01 ` Sean Christopherson
2023-08-02 14:25 ` Tom Lendacky
2023-08-02 14:33 ` Tom Lendacky
2023-08-02 15:04 ` Sean Christopherson
2023-08-02 15:26 ` Tom Lendacky
2023-08-02 15:35 ` Sean Christopherson
2023-08-02 20:03 ` Tom Lendacky
2023-08-03 3:27 ` Wu Zongyo
2023-08-03 8:44 ` Wu Zongyo
2023-08-03 14:34 ` Sean Christopherson
2023-08-04 2:33 ` Wu Zongyo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8eb933fd-2cf3-d7a9-32fe-2a1d82eac42a@mail.ustc.edu.cn \
--to=wuzongyo@mail.ustc.edu.cn \
--cc=kvm@vger.kernel.org \
--cc=linux-coco@lists.linux.dev \
--cc=linux-kernel@vger.kernel.org \
--cc=thomas.lendacky@amd.com \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).