From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 83956C48BD1 for ; Fri, 11 Jun 2021 16:42:14 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 6BABB613BC for ; Fri, 11 Jun 2021 16:42:14 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S231255AbhFKQoK (ORCPT ); Fri, 11 Jun 2021 12:44:10 -0400 Received: from mga18.intel.com ([134.134.136.126]:54683 "EHLO mga18.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231179AbhFKQoB (ORCPT ); Fri, 11 Jun 2021 12:44:01 -0400 IronPort-SDR: Gdz1vdmgFjDjbFZJ6GDROUpIGqFljlHZst49fDg/aFbGj0OwuNfp5Dw9M7QhpS2UFPLnGeiBXd z5mpvr260nNw== X-IronPort-AV: E=McAfee;i="6200,9189,10012"; a="192872378" X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="192872378" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jun 2021 09:42:02 -0700 IronPort-SDR: jRJbQjuwfx1t2SjicXof0uL+86V0gQ2cRviybXysdyuAWE8PeWQBtmW2B+2FKAvozv7FQI98M8 NyxVf5lT3sYA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="441608132" Received: from viggo.jf.intel.com (HELO localhost.localdomain) ([10.54.77.144]) by orsmga007.jf.intel.com with ESMTP; 11 Jun 2021 09:42:02 -0700 Subject: [PATCH 4/4] selftests/vm/pkeys: Exercise x86 XSAVE init state To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org, Dave Hansen , tglx@linutronix.de, linuxram@us.ibm.com, sandipan@linux.ibm.com, akpm@linux-foundation.org, fweimer@redhat.com, desnesn@linux.vnet.ibm.com, mingo@kernel.org, bauerman@linux.ibm.com, aneesh.kumar@linux.ibm.com, mpe@ellerman.id.au, mhocko@kernel.org, msuchanek@suse.de, shuah@kernel.org, x86@kernel.org From: Dave Hansen Date: Fri, 11 Jun 2021 09:42:02 -0700 References: <20210611164153.91B76FB8@viggo.jf.intel.com> In-Reply-To: <20210611164153.91B76FB8@viggo.jf.intel.com> Message-Id: <20210611164202.1849B712@viggo.jf.intel.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Dave Hansen On x86, there is a set of instructions used to save and restore register state collectively known as the XSAVE architecture. There are about a dozen different features managed with XSAVE. The protection keys register, PKRU, is one of those features. The hardware optimizes XSAVE by tracking when the state has not changed from its initial (init) state. In this case, it can avoid the cost of writing state to memory (it would usually just be a bunch of 0's). When the pkey register is 0x0 the hardware optionally choose to track the register as being in the init state (optimize away the writes). AMD CPUs do this more aggressively compared to Intel. On x86, PKRU is rarely in its (very permissive) init state. Instead, the value defaults to something very restrictive. It is not surprising that bugs have popped up in the rare cases when PKRU reaches its init state. Add a protection key selftest which gets the protection keys register into its init state in a way that should work on Intel and AMD. Then, do a bunch of pkey register reads to watch for inadvertent changes. This adds "-mxsave" to CFLAGS for all the x86 vm selftests in order to allow use of the XSAVE instruction __builtin functions. This will make the builtins available on all of the vm selftests, but is expected to be harmless. Signed-off-by: Dave Hansen Signed-off-by: Thomas Gleixner Cc: Ram Pai Cc: Sandipan Das Cc: Andrew Morton Cc: Florian Weimer Cc: "Desnes A. Nunes do Rosario" Cc: Ingo Molnar Cc: Thiago Jung Bauermann Cc: "Aneesh Kumar K.V" Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Suchanek Cc: Shuah Khan Cc: x86@kernel.org --- b/tools/testing/selftests/vm/Makefile | 4 - b/tools/testing/selftests/vm/pkey-x86.h | 1 b/tools/testing/selftests/vm/protection_keys.c | 73 +++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 2 deletions(-) diff -puN tools/testing/selftests/vm/Makefile~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 tools/testing/selftests/vm/Makefile --- a/tools/testing/selftests/vm/Makefile~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 2021-06-11 09:41:34.574468058 -0700 +++ b/tools/testing/selftests/vm/Makefile 2021-06-11 09:41:34.588468058 -0700 @@ -100,7 +100,7 @@ $(1) $(1)_64: $(OUTPUT)/$(1)_64 endef ifeq ($(CAN_BUILD_I386),1) -$(BINARIES_32): CFLAGS += -m32 +$(BINARIES_32): CFLAGS += -m32 -mxsave $(BINARIES_32): LDLIBS += -lrt -ldl -lm $(BINARIES_32): $(OUTPUT)/%_32: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@ @@ -108,7 +108,7 @@ $(foreach t,$(TARGETS),$(eval $(call gen endif ifeq ($(CAN_BUILD_X86_64),1) -$(BINARIES_64): CFLAGS += -m64 +$(BINARIES_64): CFLAGS += -m64 -mxsave $(BINARIES_64): LDLIBS += -lrt -ldl $(BINARIES_64): $(OUTPUT)/%_64: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@ diff -puN tools/testing/selftests/vm/pkey-x86.h~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 tools/testing/selftests/vm/pkey-x86.h --- a/tools/testing/selftests/vm/pkey-x86.h~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 2021-06-11 09:41:34.576468058 -0700 +++ b/tools/testing/selftests/vm/pkey-x86.h 2021-06-11 09:41:34.590468058 -0700 @@ -126,6 +126,7 @@ static inline u32 pkey_bit_position(int #define XSTATE_PKEY_BIT (9) #define XSTATE_PKEY 0x200 +#define XSTATE_BV_OFFSET 512 int pkey_reg_xstate_offset(void) { diff -puN tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 tools/testing/selftests/vm/protection_keys.c --- a/tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 2021-06-11 09:41:34.578468058 -0700 +++ b/tools/testing/selftests/vm/protection_keys.c 2021-06-11 09:41:34.593468058 -0700 @@ -1277,6 +1277,78 @@ void test_pkey_alloc_exhaust(int *ptr, u } } +void arch_force_pkey_reg_init(void) +{ +#if defined(__i386__) || defined(__x86_64__) /* arch */ + u64 *buf; + + /* + * All keys should be allocated and set to allow reads and + * writes, so the register should be all 0. If not, just + * skip the test. + */ + if (read_pkey_reg()) + return; + + /* + * Just allocate an absurd about of memory rather than + * doing the XSAVE size enumeration dance. + */ + buf = mmap(NULL, 1*MB, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); + + /* These __builtins require compiling with -mxsave */ + + /* XSAVE to build a valid buffer: */ + __builtin_ia32_xsave(buf, XSTATE_PKEY); + /* Clear XSTATE_BV[PKRU]: */ + buf[XSTATE_BV_OFFSET/sizeof(u64)] &= ~XSTATE_PKEY; + /* XRSTOR will likely get PKRU back to the init state: */ + __builtin_ia32_xrstor(buf, XSTATE_PKEY); + + munmap(buf, 1*MB); +#endif +} + + +/* + * This is mostly useless on ppc for now. But it will not + * hurt anything and should give some better coverage as + * a long-running test that continually checks the pkey + * register. + */ +void test_pkey_init_state(int *ptr, u16 pkey) +{ + int err; + int allocated_pkeys[NR_PKEYS] = {0}; + int nr_allocated_pkeys = 0; + int i; + + for (i = 0; i < NR_PKEYS; i++) { + int new_pkey = alloc_pkey(); + + if (new_pkey < 0) + continue; + allocated_pkeys[nr_allocated_pkeys++] = new_pkey; + } + + dprintf3("%s()::%d\n", __func__, __LINE__); + + arch_force_pkey_reg_init(); + + /* + * Loop for a bit, hoping to get exercise the kernel + * context switch code. + */ + for (i = 0; i < 1000000; i++) + read_pkey_reg(); + + for (i = 0; i < nr_allocated_pkeys; i++) { + err = sys_pkey_free(allocated_pkeys[i]); + pkey_assert(!err); + read_pkey_reg(); /* for shadow checking */ + } +} + /* * pkey 0 is special. It is allocated by default, so you do not * have to call pkey_alloc() to use it first. Make sure that it @@ -1508,6 +1580,7 @@ void (*pkey_tests[])(int *ptr, u16 pkey) test_implicit_mprotect_exec_only_memory, test_mprotect_with_pkey_0, test_ptrace_of_child, + test_pkey_init_state, test_pkey_syscalls_on_non_allocated_pkey, test_pkey_syscalls_bad_args, test_pkey_alloc_exhaust, _ From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-13.8 required=3.0 tests=BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_CR_TRAILER,INCLUDES_PATCH, MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,URIBL_RED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id 72A33C48BE0 for ; Fri, 11 Jun 2021 16:42:13 +0000 (UTC) Received: from kanga.kvack.org (kanga.kvack.org [205.233.56.17]) by mail.kernel.org (Postfix) with ESMTP id DA3D3613BA for ; Fri, 11 Jun 2021 16:42:12 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org DA3D3613BA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=linux.intel.com Authentication-Results: mail.kernel.org; spf=pass smtp.mailfrom=owner-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix) id 7D3E46B0072; Fri, 11 Jun 2021 12:42:12 -0400 (EDT) Received: by kanga.kvack.org (Postfix, from userid 40) id 7AB6B6B0073; Fri, 11 Jun 2021 12:42:12 -0400 (EDT) X-Delivered-To: int-list-linux-mm@kvack.org Received: by kanga.kvack.org (Postfix, from userid 63042) id 64B706B0074; Fri, 11 Jun 2021 12:42:12 -0400 (EDT) X-Delivered-To: linux-mm@kvack.org Received: from forelay.hostedemail.com (smtprelay0034.hostedemail.com [216.40.44.34]) by kanga.kvack.org (Postfix) with ESMTP id 3554E6B0072 for ; Fri, 11 Jun 2021 12:42:12 -0400 (EDT) Received: from smtpin20.hostedemail.com (10.5.19.251.rfc1918.com [10.5.19.251]) by forelay04.hostedemail.com (Postfix) with ESMTP id CD242BBE8 for ; Fri, 11 Jun 2021 16:42:11 +0000 (UTC) X-FDA: 78242010462.20.A786DEB Received: from mga05.intel.com (mga05.intel.com [192.55.52.43]) by imf28.hostedemail.com (Postfix) with ESMTP id 75B66200107A for ; Fri, 11 Jun 2021 16:42:05 +0000 (UTC) IronPort-SDR: fu7U1HbvysPSVmJjVawyh4SvxvAHCzIXZ5tKQBGmxbNmxkl8T9MxAesKrNntWKf8Uw+XppU7V4 OwtRUc0paVSA== X-IronPort-AV: E=McAfee;i="6200,9189,10012"; a="291192804" X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="291192804" Received: from orsmga007.jf.intel.com ([10.7.209.58]) by fmsmga105.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 11 Jun 2021 09:42:03 -0700 IronPort-SDR: jRJbQjuwfx1t2SjicXof0uL+86V0gQ2cRviybXysdyuAWE8PeWQBtmW2B+2FKAvozv7FQI98M8 NyxVf5lT3sYA== X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.83,265,1616482800"; d="scan'208";a="441608132" Received: from viggo.jf.intel.com (HELO localhost.localdomain) ([10.54.77.144]) by orsmga007.jf.intel.com with ESMTP; 11 Jun 2021 09:42:02 -0700 Subject: [PATCH 4/4] selftests/vm/pkeys: Exercise x86 XSAVE init state To: linux-mm@kvack.org Cc: linux-kernel@vger.kernel.org,Dave Hansen ,tglx@linutronix.de,linuxram@us.ibm.com,sandipan@linux.ibm.com,akpm@linux-foundation.org,fweimer@redhat.com,desnesn@linux.vnet.ibm.com,mingo@kernel.org,bauerman@linux.ibm.com,aneesh.kumar@linux.ibm.com,mpe@ellerman.id.au,mhocko@kernel.org,msuchanek@suse.de,shuah@kernel.org,x86@kernel.org From: Dave Hansen Date: Fri, 11 Jun 2021 09:42:02 -0700 References: <20210611164153.91B76FB8@viggo.jf.intel.com> In-Reply-To: <20210611164153.91B76FB8@viggo.jf.intel.com> Message-Id: <20210611164202.1849B712@viggo.jf.intel.com> Authentication-Results: imf28.hostedemail.com; dkim=none; dmarc=fail reason="No valid SPF, No valid DKIM" header.from=intel.com (policy=none); spf=none (imf28.hostedemail.com: domain of dave.hansen@linux.intel.com has no SPF policy when checking 192.55.52.43) smtp.mailfrom=dave.hansen@linux.intel.com X-Stat-Signature: d96jiqctooi847fs4jqwmj3n5xnci4i5 X-Rspamd-Queue-Id: 75B66200107A X-Rspamd-Server: rspam06 X-HE-Tag: 1623429725-664676 X-Bogosity: Ham, tests=bogofilter, spamicity=0.000000, version=1.2.4 Sender: owner-linux-mm@kvack.org Precedence: bulk X-Loop: owner-majordomo@kvack.org List-ID: From: Dave Hansen On x86, there is a set of instructions used to save and restore register state collectively known as the XSAVE architecture. There are about a dozen different features managed with XSAVE. The protection keys register, PKRU, is one of those features. The hardware optimizes XSAVE by tracking when the state has not changed from its initial (init) state. In this case, it can avoid the cost of writing state to memory (it would usually just be a bunch of 0's). When the pkey register is 0x0 the hardware optionally choose to track the register as being in the init state (optimize away the writes). AMD CPUs do this more aggressively compared to Intel. On x86, PKRU is rarely in its (very permissive) init state. Instead, the value defaults to something very restrictive. It is not surprising that bugs have popped up in the rare cases when PKRU reaches its init state. Add a protection key selftest which gets the protection keys register into its init state in a way that should work on Intel and AMD. Then, do a bunch of pkey register reads to watch for inadvertent changes. This adds "-mxsave" to CFLAGS for all the x86 vm selftests in order to allow use of the XSAVE instruction __builtin functions. This will make the builtins available on all of the vm selftests, but is expected to be harmless. Signed-off-by: Dave Hansen Signed-off-by: Thomas Gleixner Cc: Ram Pai Cc: Sandipan Das Cc: Andrew Morton Cc: Florian Weimer Cc: "Desnes A. Nunes do Rosario" Cc: Ingo Molnar Cc: Thiago Jung Bauermann Cc: "Aneesh Kumar K.V" Cc: Michael Ellerman Cc: Michal Hocko Cc: Michal Suchanek Cc: Shuah Khan Cc: x86@kernel.org --- b/tools/testing/selftests/vm/Makefile | 4 - b/tools/testing/selftests/vm/pkey-x86.h | 1 b/tools/testing/selftests/vm/protection_keys.c | 73 +++++++++++++++++++++++++ 3 files changed, 76 insertions(+), 2 deletions(-) diff -puN tools/testing/selftests/vm/Makefile~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 tools/testing/selftests/vm/Makefile --- a/tools/testing/selftests/vm/Makefile~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 2021-06-11 09:41:34.574468058 -0700 +++ b/tools/testing/selftests/vm/Makefile 2021-06-11 09:41:34.588468058 -0700 @@ -100,7 +100,7 @@ $(1) $(1)_64: $(OUTPUT)/$(1)_64 endef ifeq ($(CAN_BUILD_I386),1) -$(BINARIES_32): CFLAGS += -m32 +$(BINARIES_32): CFLAGS += -m32 -mxsave $(BINARIES_32): LDLIBS += -lrt -ldl -lm $(BINARIES_32): $(OUTPUT)/%_32: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@ @@ -108,7 +108,7 @@ $(foreach t,$(TARGETS),$(eval $(call gen endif ifeq ($(CAN_BUILD_X86_64),1) -$(BINARIES_64): CFLAGS += -m64 +$(BINARIES_64): CFLAGS += -m64 -mxsave $(BINARIES_64): LDLIBS += -lrt -ldl $(BINARIES_64): $(OUTPUT)/%_64: %.c $(CC) $(CFLAGS) $(EXTRA_CFLAGS) $(notdir $^) $(LDLIBS) -o $@ diff -puN tools/testing/selftests/vm/pkey-x86.h~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 tools/testing/selftests/vm/pkey-x86.h --- a/tools/testing/selftests/vm/pkey-x86.h~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 2021-06-11 09:41:34.576468058 -0700 +++ b/tools/testing/selftests/vm/pkey-x86.h 2021-06-11 09:41:34.590468058 -0700 @@ -126,6 +126,7 @@ static inline u32 pkey_bit_position(int #define XSTATE_PKEY_BIT (9) #define XSTATE_PKEY 0x200 +#define XSTATE_BV_OFFSET 512 int pkey_reg_xstate_offset(void) { diff -puN tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 tools/testing/selftests/vm/protection_keys.c --- a/tools/testing/selftests/vm/protection_keys.c~selftests_vm_pkeys_Exercise_x86_XSAVE_init_state-1 2021-06-11 09:41:34.578468058 -0700 +++ b/tools/testing/selftests/vm/protection_keys.c 2021-06-11 09:41:34.593468058 -0700 @@ -1277,6 +1277,78 @@ void test_pkey_alloc_exhaust(int *ptr, u } } +void arch_force_pkey_reg_init(void) +{ +#if defined(__i386__) || defined(__x86_64__) /* arch */ + u64 *buf; + + /* + * All keys should be allocated and set to allow reads and + * writes, so the register should be all 0. If not, just + * skip the test. + */ + if (read_pkey_reg()) + return; + + /* + * Just allocate an absurd about of memory rather than + * doing the XSAVE size enumeration dance. + */ + buf = mmap(NULL, 1*MB, PROT_READ|PROT_WRITE, MAP_ANONYMOUS|MAP_PRIVATE, -1, 0); + + /* These __builtins require compiling with -mxsave */ + + /* XSAVE to build a valid buffer: */ + __builtin_ia32_xsave(buf, XSTATE_PKEY); + /* Clear XSTATE_BV[PKRU]: */ + buf[XSTATE_BV_OFFSET/sizeof(u64)] &= ~XSTATE_PKEY; + /* XRSTOR will likely get PKRU back to the init state: */ + __builtin_ia32_xrstor(buf, XSTATE_PKEY); + + munmap(buf, 1*MB); +#endif +} + + +/* + * This is mostly useless on ppc for now. But it will not + * hurt anything and should give some better coverage as + * a long-running test that continually checks the pkey + * register. + */ +void test_pkey_init_state(int *ptr, u16 pkey) +{ + int err; + int allocated_pkeys[NR_PKEYS] = {0}; + int nr_allocated_pkeys = 0; + int i; + + for (i = 0; i < NR_PKEYS; i++) { + int new_pkey = alloc_pkey(); + + if (new_pkey < 0) + continue; + allocated_pkeys[nr_allocated_pkeys++] = new_pkey; + } + + dprintf3("%s()::%d\n", __func__, __LINE__); + + arch_force_pkey_reg_init(); + + /* + * Loop for a bit, hoping to get exercise the kernel + * context switch code. + */ + for (i = 0; i < 1000000; i++) + read_pkey_reg(); + + for (i = 0; i < nr_allocated_pkeys; i++) { + err = sys_pkey_free(allocated_pkeys[i]); + pkey_assert(!err); + read_pkey_reg(); /* for shadow checking */ + } +} + /* * pkey 0 is special. It is allocated by default, so you do not * have to call pkey_alloc() to use it first. Make sure that it @@ -1508,6 +1580,7 @@ void (*pkey_tests[])(int *ptr, u16 pkey) test_implicit_mprotect_exec_only_memory, test_mprotect_with_pkey_0, test_ptrace_of_child, + test_pkey_init_state, test_pkey_syscalls_on_non_allocated_pkey, test_pkey_syscalls_bad_args, test_pkey_alloc_exhaust, _