From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pj1-f42.google.com (mail-pj1-f42.google.com [209.85.216.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A7BB405C1 for ; Fri, 29 Mar 2024 07:24:45 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.216.42 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711697087; cv=none; b=ujhvkkwfctTEn1zKd9kYVtyWrv+mDFXFNXoEsfeYuzVXG5p0JgMLgVlvDZGkfD2yYle9QEtuI9rsveMD/SMRwDvRXl+K0ono2Og0X66e6Vrxs4iumRNHcKPf9erC6nxxBHwRRRPkCKNBkl09bfQVtLTODHM2cJavZbbf2lcIz5s= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1711697087; c=relaxed/simple; bh=DdPDxMPJzY9UPOgh231N4URq40JyNA4tqlto5EmYAEk=; h=From:To:Cc:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version; b=ETJWhDfxcXqwzc9f1iScV+3DFuw6518LizabI6kmdsuAwOs2xFHJZmT5hvZFue2pCzFbz/AZ0Nu2fh+a6lwnsef/z8Da7yvQeGmaX++K4sq2SDVsi1dn2hdorLWDd2naH4e0jrGJt/9js56XjPMRCSa8iCeD/6Hy1DVHk1l5pls= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com; spf=pass smtp.mailfrom=sifive.com; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b=YDpl7Xg8; arc=none smtp.client-ip=209.85.216.42 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=sifive.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=sifive.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=sifive.com header.i=@sifive.com header.b="YDpl7Xg8" Received: by mail-pj1-f42.google.com with SMTP id 98e67ed59e1d1-2a20e595351so643814a91.3 for ; Fri, 29 Mar 2024 00:24:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711697085; x=1712301885; darn=vger.kernel.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=YDpl7Xg8hvXaoL1x4XPslOT0Ape50IfZUAgH9UOwqgi43tUdEhpnn4G+EtdKlTA3zX 7CtvP616QVvD6GjPRymoQBr/F+OMKwEst5gkswwe1H9e57tlCqQPzeH+C/5hkSsz3x1S fY9W1joUVBgdqye9/N7KSr7SvsT5+rJ+AA9/Ks5WeE+pmoeOuqFyfIFUN+3xU5wD2Kn0 RApKB9K4QfbyG65nq5pDHOOGZKku3buG16/Jpm9HX4BFHZO1SHFgwt+hpaDphKTIRzIc WaEAIKKvlsglLaluvM3pgSJdmvXVB0JEInSB6htD4kLDAxUqb97UlTKb512kriOhTiP1 JWhg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711697085; x=1712301885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=qHkMKTax0MKyPuVj8GDsR8fXgBjDODGLw3FQHfDZYmFdmwjRYBHIZPxN7HWzH2Qjyo lfbxZ+19LJlnV8cRvyazt2U5H6Y7lWER0SG7Efyuue5jqYy+4t2l8jfRMA9IotphcycY TSFKJszI2EpNgWAFCyNb/pmtWV6jcRFE2Cv/G42PrfKY0uOUIGsUQZySh2P0MknHplz/ Yg0kKQrKZwJ0WXIrzljX3C+eBP7VA8y7eGo9yX51yORm37eyFcctxKmzgY50mqB3VZf8 ag0wSudSV/RaLbaYV+Kjh1yRg4veMNtlFiu9vP5H8BQlPRQNeVQoiN2sZgiCoxwSuesU P9sg== X-Gm-Message-State: AOJu0YyzWqhP+Y2X/WDCZGGhdg7xizslliUa+jVZgLiDjcZ3cOcGTOy4 AQIcuiv3XYj7KZJlgXgyzRn0riYbxDr8GL5fa1c9SxC5Ey50+GDvfvEf4Riv1XI= X-Google-Smtp-Source: AGHT+IHa/zXusbJGA4K8cLPLAWn+CnPIA4cHPz+eGb6xCnwX+ovRVMEI/wA60Ad/6sOmnhgR0Mc6ng== X-Received: by 2002:a17:90b:3c50:b0:2a2:19f:dbc7 with SMTP id pm16-20020a17090b3c5000b002a2019fdbc7mr1989065pjb.0.1711697084840; Fri, 29 Mar 2024 00:24:44 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id b8-20020a17090a010800b0029ddac03effsm4971798pjb.11.2024.03.29.00.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Mar 2024 00:24:44 -0700 (PDT) From: Samuel Holland To: Andrew Morton , linux-arm-kernel@lists.infradead.org, x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Christoph Hellwig , loongarch@lists.linux.dev, amd-gfx@lists.freedesktop.org, Samuel Holland , Borislav Petkov , Catalin Marinas , Dave Hansen , Huacai Chen , Ingo Molnar , Jonathan Corbet , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Russell King , Thomas Gleixner , Will Deacon , linux-doc@vger.kernel.org, linux-kbuild@vger.kernel.org Subject: [PATCH v4 01/15] arch: Add ARCH_HAS_KERNEL_FPU_SUPPORT Date: Fri, 29 Mar 2024 00:18:16 -0700 Message-ID: <20240329072441.591471-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240329072441.591471-1-samuel.holland@sifive.com> References: <20240329072441.591471-1-samuel.holland@sifive.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Several architectures provide an API to enable the FPU and run floating-point SIMD code in kernel space. However, the function names, header locations, and semantics are inconsistent across architectures, and FPU support may be gated behind other Kconfig options. Provide a standard way for architectures to declare that kernel space FPU support is available. Architectures selecting this option must implement what is currently the most common API (kernel_fpu_begin() and kernel_fpu_end(), plus a new function kernel_fpu_available()) and provide the appropriate CFLAGS for compiling floating-point C code. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Add documentation explaining the built-time and runtime APIs - Add a linux/fpu.h header for generic isolation enforcement Documentation/core-api/floating-point.rst | 78 +++++++++++++++++++++++ Documentation/core-api/index.rst | 1 + Makefile | 5 ++ arch/Kconfig | 6 ++ include/linux/fpu.h | 12 ++++ 5 files changed, 102 insertions(+) create mode 100644 Documentation/core-api/floating-point.rst create mode 100644 include/linux/fpu.h diff --git a/Documentation/core-api/floating-point.rst b/Documentation/core-api/floating-point.rst new file mode 100644 index 000000000000..a8d0d4b05052 --- /dev/null +++ b/Documentation/core-api/floating-point.rst @@ -0,0 +1,78 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +Floating-point API +================== + +Kernel code is normally prohibited from using floating-point (FP) registers or +instructions, including the C float and double data types. This rule reduces +system call overhead, because the kernel does not need to save and restore the +userspace floating-point register state. + +However, occasionally drivers or library functions may need to include FP code. +This is supported by isolating the functions containing FP code to a separate +translation unit (a separate source file), and saving/restoring the FP register +state around calls to those functions. This creates "critical sections" of +floating-point usage. + +The reason for this isolation is to prevent the compiler from generating code +touching the FP registers outside these critical sections. Compilers sometimes +use FP registers to optimize inlined ``memcpy`` or variable assignment, as +floating-point registers may be wider than general-purpose registers. + +Usability of floating-point code within the kernel is architecture-specific. +Additionally, because a single kernel may be configured to support platforms +both with and without a floating-point unit, FPU availability must be checked +both at build time and at run time. + +Several architectures implement the generic kernel floating-point API from +``linux/fpu.h``, as described below. Some other architectures implement their +own unique APIs, which are documented separately. + +Build-time API +-------------- + +Floating-point code may be built if the option ``ARCH_HAS_KERNEL_FPU_SUPPORT`` +is enabled. For C code, such code must be placed in a separate file, and that +file must have its compilation flags adjusted using the following pattern:: + + CFLAGS_foo.o += $(CC_FLAGS_FPU) + CFLAGS_REMOVE_foo.o += $(CC_FLAGS_NO_FPU) + +Architectures are expected to define one or both of these variables in their +top-level Makefile as needed. For example:: + + CC_FLAGS_FPU := -mhard-float + +or:: + + CC_FLAGS_NO_FPU := -msoft-float + +Normal kernel code is assumed to use the equivalent of ``CC_FLAGS_NO_FPU``. + +Runtime API +----------- + +The runtime API is provided in ``linux/fpu.h``. This header cannot be included +from files implementing FP code (those with their compilation flags adjusted as +above). Instead, it must be included when defining the FP critical sections. + +.. c:function:: bool kernel_fpu_available( void ) + + This function reports if floating-point code can be used on this CPU or + platform. The value returned by this function is not expected to change + at runtime, so it only needs to be called once, not before every + critical section. + +.. c:function:: void kernel_fpu_begin( void ) + void kernel_fpu_end( void ) + + These functions create a floating-point critical section. It is only + valid to call ``kernel_fpu_begin()`` after a previous call to + ``kernel_fpu_available()`` returned ``true``. These functions are only + guaranteed to be callable from (preemptible or non-preemptible) process + context. + + Preemption may be disabled inside critical sections, so their size + should be minimized. They are *not* required to be reentrant. If the + caller expects to nest critical sections, it must implement its own + reference counting. diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index 7a3a08d81f11..974beccd671f 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -48,6 +48,7 @@ Library functionality that is used throughout the kernel. errseq wrappers/atomic_t wrappers/atomic_bitops + floating-point Low level entry and exit ======================== diff --git a/Makefile b/Makefile index 763b6792d3d5..710f65e4249d 100644 --- a/Makefile +++ b/Makefile @@ -964,6 +964,11 @@ KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif +# Architectures can define flags to add/remove for floating-point support +CC_FLAGS_FPU += -D_LINUX_FPU_COMPILATION_UNIT +export CC_FLAGS_FPU +export CC_FLAGS_NO_FPU + ifneq ($(CONFIG_FUNCTION_ALIGNMENT),0) # Set the minimal function alignment. Use the newer GCC option # -fmin-function-alignment if it is available, or fall back to -falign-funtions. diff --git a/arch/Kconfig b/arch/Kconfig index 9f066785bb71..8e34b3acf73d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1569,6 +1569,12 @@ config ARCH_HAS_NONLEAF_PMD_YOUNG address translations. Page table walkers that clear the accessed bit may use this capability to reduce their search space. +config ARCH_HAS_KERNEL_FPU_SUPPORT + bool + help + Architectures that select this option can run floating-point code in + the kernel, as described in Documentation/core-api/floating-point.rst. + source "kernel/gcov/Kconfig" source "scripts/gcc-plugins/Kconfig" diff --git a/include/linux/fpu.h b/include/linux/fpu.h new file mode 100644 index 000000000000..2fb63e22913b --- /dev/null +++ b/include/linux/fpu.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _LINUX_FPU_H +#define _LINUX_FPU_H + +#ifdef _LINUX_FPU_COMPILATION_UNIT +#error FP code must be compiled separately. See Documentation/core-api/floating-point.rst. +#endif + +#include + +#endif -- 2.44.0 From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B1B1AC6FD1F for ; Fri, 29 Mar 2024 07:25:11 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=roIOzpI1yZEgsmjBu1jfh9l1iTci2iezLO4rw5Jqv5A=; b=DqukMDFCDm/J1a dIJ/spaYO/PLRx0m0ZausMDOYhSuUaQ7MmX4c4We1f8ndymBYQIrVFcWmz+sBvnUYbwdCpJHI1K1+ /iA7nWN3weS2MIegp61y48zuVIC1kN5VnEnFkkbBKupuWHcB0Mr0KFOzxAEAJiTrJx6iQGNqymqpf KF0rCmhRcBr0Rlzxv0GKNZLi+NqYzX+nBAHrO+HvqtSnYU5sok69CQKG94SiT2xZrcD27IFEfCyts 1HmxcIQN5fEnQaPJJOeEiDvFT7RzhxOCm9VhkXnZtJZ1xi4pwoyFBuUqkjDeBWIgFcL6yeBTQkO+0 MF7VTFF3vo6haaWuwPBA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rq6bn-0000000H8he-2qFg; Fri, 29 Mar 2024 07:25:03 +0000 Received: from mail-pj1-x1035.google.com ([2607:f8b0:4864:20::1035]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rq6bW-0000000H8WP-1K0h for linux-riscv@lists.infradead.org; Fri, 29 Mar 2024 07:24:50 +0000 Received: by mail-pj1-x1035.google.com with SMTP id 98e67ed59e1d1-29f9af2e0b7so1255768a91.1 for ; Fri, 29 Mar 2024 00:24:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711697085; x=1712301885; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=Ph+OaZC9B/epPwwZr4L9xkFW4V0Q6pLnEdaQVcKpxv3niyxrlzpQrHVIf3qSD6thtF NbZaKnak7KOeIIAtwn6B3oTPFZUNS0obCzRUI90MI0jSixIFuYiXT42+Q2t/Ce34o99i Vt61P8+IDfZMOl/9uowicJHG5Lr5ajMQ6W7ZlQhzv4wuTMYvpf6X3r33Tws73VRxMWri kRciWNif1Sm7ysj9YvFraHTijX17nWZ0HUbbPr2JXMAh01oI7ReXvruBN29Mutaj2lLA ABASaVGfGJy8DEzwm/0M8W4kkuTpzUjlUwbysbz+lSaUkq95HR6U4kKrHsMRYEhQJkd/ /X5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711697085; x=1712301885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=UsMK6hNE+jCEHcoBQXs9Tq/q0loTT6EJthAhMtwS8GfObXmZvXosGHqW1MR8mVKeo4 8T3KyZyOItUaRKyN3fcOeGm+6C0dePkWGfcxr+2RGFf8wAqK8po+3oB34frW1LGI6XZA EvTUPBZT7WHsJzwUR96hjdz3CXk19VYyN3U3bYYJRbfPhhHtgIegrP37ntCcyoxOVVSo ZywHKcxgCMta04vmVtpclP2nraaRkwmM4zNiA/+j56dJO3RDvcOBBqLsk9dGgg5ZGe3Z s8+K2yhqQR4sVRnOTNqXj7kIAtkJz9Z2B97B3TkvQB5wDhtmEKXFcwt9nKlvq5dkuF5E 2zjQ== X-Forwarded-Encrypted: i=1; AJvYcCUjmfy95Z9mTXu7wzSAqsigzmBwY2PRigxvFOpDhCXAPzCqsZOlAjvCztFGPI2tseZ9kd4j0dMFVVplD5xQNCxLrv8SjOE58MafFCHUo77h X-Gm-Message-State: AOJu0Yy9623Ra79OawvGK+UKRycx7urIzmJaNsBotTF30FgcKIfr+g1x MmyryzqsX8s4AsFYxG2+lBj4Qj15Amc59+WLKFGzMkNH0nNNT6X2yUUqSECqFUU= X-Google-Smtp-Source: AGHT+IHa/zXusbJGA4K8cLPLAWn+CnPIA4cHPz+eGb6xCnwX+ovRVMEI/wA60Ad/6sOmnhgR0Mc6ng== X-Received: by 2002:a17:90b:3c50:b0:2a2:19f:dbc7 with SMTP id pm16-20020a17090b3c5000b002a2019fdbc7mr1989065pjb.0.1711697084840; Fri, 29 Mar 2024 00:24:44 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id b8-20020a17090a010800b0029ddac03effsm4971798pjb.11.2024.03.29.00.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Mar 2024 00:24:44 -0700 (PDT) From: Samuel Holland To: Andrew Morton , linux-arm-kernel@lists.infradead.org, x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Christoph Hellwig , loongarch@lists.linux.dev, amd-gfx@lists.freedesktop.org, Samuel Holland , Borislav Petkov , Catalin Marinas , Dave Hansen , Huacai Chen , Ingo Molnar , Jonathan Corbet , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Russell King , Thomas Gleixner , Will Deacon , linux-doc@vger.kernel.org, linux-kbuild@vger.kernel.org Subject: [PATCH v4 01/15] arch: Add ARCH_HAS_KERNEL_FPU_SUPPORT Date: Fri, 29 Mar 2024 00:18:16 -0700 Message-ID: <20240329072441.591471-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240329072441.591471-1-samuel.holland@sifive.com> References: <20240329072441.591471-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240329_002446_467786_077AEDB6 X-CRM114-Status: GOOD ( 29.35 ) X-BeenThere: linux-riscv@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-riscv" Errors-To: linux-riscv-bounces+linux-riscv=archiver.kernel.org@lists.infradead.org Several architectures provide an API to enable the FPU and run floating-point SIMD code in kernel space. However, the function names, header locations, and semantics are inconsistent across architectures, and FPU support may be gated behind other Kconfig options. Provide a standard way for architectures to declare that kernel space FPU support is available. Architectures selecting this option must implement what is currently the most common API (kernel_fpu_begin() and kernel_fpu_end(), plus a new function kernel_fpu_available()) and provide the appropriate CFLAGS for compiling floating-point C code. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Add documentation explaining the built-time and runtime APIs - Add a linux/fpu.h header for generic isolation enforcement Documentation/core-api/floating-point.rst | 78 +++++++++++++++++++++++ Documentation/core-api/index.rst | 1 + Makefile | 5 ++ arch/Kconfig | 6 ++ include/linux/fpu.h | 12 ++++ 5 files changed, 102 insertions(+) create mode 100644 Documentation/core-api/floating-point.rst create mode 100644 include/linux/fpu.h diff --git a/Documentation/core-api/floating-point.rst b/Documentation/core-api/floating-point.rst new file mode 100644 index 000000000000..a8d0d4b05052 --- /dev/null +++ b/Documentation/core-api/floating-point.rst @@ -0,0 +1,78 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +Floating-point API +================== + +Kernel code is normally prohibited from using floating-point (FP) registers or +instructions, including the C float and double data types. This rule reduces +system call overhead, because the kernel does not need to save and restore the +userspace floating-point register state. + +However, occasionally drivers or library functions may need to include FP code. +This is supported by isolating the functions containing FP code to a separate +translation unit (a separate source file), and saving/restoring the FP register +state around calls to those functions. This creates "critical sections" of +floating-point usage. + +The reason for this isolation is to prevent the compiler from generating code +touching the FP registers outside these critical sections. Compilers sometimes +use FP registers to optimize inlined ``memcpy`` or variable assignment, as +floating-point registers may be wider than general-purpose registers. + +Usability of floating-point code within the kernel is architecture-specific. +Additionally, because a single kernel may be configured to support platforms +both with and without a floating-point unit, FPU availability must be checked +both at build time and at run time. + +Several architectures implement the generic kernel floating-point API from +``linux/fpu.h``, as described below. Some other architectures implement their +own unique APIs, which are documented separately. + +Build-time API +-------------- + +Floating-point code may be built if the option ``ARCH_HAS_KERNEL_FPU_SUPPORT`` +is enabled. For C code, such code must be placed in a separate file, and that +file must have its compilation flags adjusted using the following pattern:: + + CFLAGS_foo.o += $(CC_FLAGS_FPU) + CFLAGS_REMOVE_foo.o += $(CC_FLAGS_NO_FPU) + +Architectures are expected to define one or both of these variables in their +top-level Makefile as needed. For example:: + + CC_FLAGS_FPU := -mhard-float + +or:: + + CC_FLAGS_NO_FPU := -msoft-float + +Normal kernel code is assumed to use the equivalent of ``CC_FLAGS_NO_FPU``. + +Runtime API +----------- + +The runtime API is provided in ``linux/fpu.h``. This header cannot be included +from files implementing FP code (those with their compilation flags adjusted as +above). Instead, it must be included when defining the FP critical sections. + +.. c:function:: bool kernel_fpu_available( void ) + + This function reports if floating-point code can be used on this CPU or + platform. The value returned by this function is not expected to change + at runtime, so it only needs to be called once, not before every + critical section. + +.. c:function:: void kernel_fpu_begin( void ) + void kernel_fpu_end( void ) + + These functions create a floating-point critical section. It is only + valid to call ``kernel_fpu_begin()`` after a previous call to + ``kernel_fpu_available()`` returned ``true``. These functions are only + guaranteed to be callable from (preemptible or non-preemptible) process + context. + + Preemption may be disabled inside critical sections, so their size + should be minimized. They are *not* required to be reentrant. If the + caller expects to nest critical sections, it must implement its own + reference counting. diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index 7a3a08d81f11..974beccd671f 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -48,6 +48,7 @@ Library functionality that is used throughout the kernel. errseq wrappers/atomic_t wrappers/atomic_bitops + floating-point Low level entry and exit ======================== diff --git a/Makefile b/Makefile index 763b6792d3d5..710f65e4249d 100644 --- a/Makefile +++ b/Makefile @@ -964,6 +964,11 @@ KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif +# Architectures can define flags to add/remove for floating-point support +CC_FLAGS_FPU += -D_LINUX_FPU_COMPILATION_UNIT +export CC_FLAGS_FPU +export CC_FLAGS_NO_FPU + ifneq ($(CONFIG_FUNCTION_ALIGNMENT),0) # Set the minimal function alignment. Use the newer GCC option # -fmin-function-alignment if it is available, or fall back to -falign-funtions. diff --git a/arch/Kconfig b/arch/Kconfig index 9f066785bb71..8e34b3acf73d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1569,6 +1569,12 @@ config ARCH_HAS_NONLEAF_PMD_YOUNG address translations. Page table walkers that clear the accessed bit may use this capability to reduce their search space. +config ARCH_HAS_KERNEL_FPU_SUPPORT + bool + help + Architectures that select this option can run floating-point code in + the kernel, as described in Documentation/core-api/floating-point.rst. + source "kernel/gcov/Kconfig" source "scripts/gcc-plugins/Kconfig" diff --git a/include/linux/fpu.h b/include/linux/fpu.h new file mode 100644 index 000000000000..2fb63e22913b --- /dev/null +++ b/include/linux/fpu.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _LINUX_FPU_H +#define _LINUX_FPU_H + +#ifdef _LINUX_FPU_COMPILATION_UNIT +#error FP code must be compiled separately. See Documentation/core-api/floating-point.rst. +#endif + +#include + +#endif -- 2.44.0 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 8D95ACD1288 for ; Fri, 29 Mar 2024 07:25:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Message-ID:Date:Subject:Cc:To:From:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=PtYrbAUwJ0Yx5hSL9nhztOlrrMa97yHSAuuJzxEMjws=; b=UFNU3vQZK96wRB 469uoZs+sJWwiIC7X1A1sXgXgroPM5cAHHpgU1JBMtcfYGnIvan2FsXl5rFBrZHcC1uRRAqRICHef 8qyAfxLny2CNvAdVEebSimnX0CCWrAe3p1zuGGaXLzyHb+57L7YmQ5++wkIIDsyNIKbCmQBDwm+4p CpWVrsF6BQYqev6TqYSGYYb/GojqADTAHUsfa3eqJbRJeTeVnx7GFPgx/z0wbG1Lyw6aHR7TPBlN7 FVT5xtqZ3sBV5qiFlS4igCJclpj4gphEE6Hzo+528G3MvWY0Q7ZPf+rNlnGnRJWq9k/3HlqH1FvKe NAYAaLCbu2mdG51kqRqA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1rq6bm-0000000H8gN-0O4r; Fri, 29 Mar 2024 07:25:02 +0000 Received: from mail-pj1-x1030.google.com ([2607:f8b0:4864:20::1030]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1rq6bW-0000000H8WQ-1Mc2 for linux-arm-kernel@lists.infradead.org; Fri, 29 Mar 2024 07:24:49 +0000 Received: by mail-pj1-x1030.google.com with SMTP id 98e67ed59e1d1-2a21330d093so347424a91.2 for ; Fri, 29 Mar 2024 00:24:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711697085; x=1712301885; darn=lists.infradead.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=Ph+OaZC9B/epPwwZr4L9xkFW4V0Q6pLnEdaQVcKpxv3niyxrlzpQrHVIf3qSD6thtF NbZaKnak7KOeIIAtwn6B3oTPFZUNS0obCzRUI90MI0jSixIFuYiXT42+Q2t/Ce34o99i Vt61P8+IDfZMOl/9uowicJHG5Lr5ajMQ6W7ZlQhzv4wuTMYvpf6X3r33Tws73VRxMWri kRciWNif1Sm7ysj9YvFraHTijX17nWZ0HUbbPr2JXMAh01oI7ReXvruBN29Mutaj2lLA ABASaVGfGJy8DEzwm/0M8W4kkuTpzUjlUwbysbz+lSaUkq95HR6U4kKrHsMRYEhQJkd/ /X5A== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711697085; x=1712301885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=qp5xZiMdokAk7NwyvaoInAgek9Xt6v1Ia9gWGRX3fmQBc46891nfffKBoONGYdIz3U pByRuemtWO8m6ZD6DLhZJMUX9p12M0ISqoBlaZPLdA1XHxWh8A+YcULZk1VywADBvNa3 Rc1KQm8DwgxHpqO/TAfftq7THseCdMJsBGZZINULblnNvEbZzjYeKPk+NY44ERRZGrs+ zsHAm7vN6oRcD+4qGMBZbttN9VoHzkBMvoOIgIL5sZHmk8JRAWL/5Vg3pGj8fgiDH8aL Mr3uOFJCLzwy+Gr4fjINqng5iRd8j1ZnZIUpmVqcMaHWmSEPuFJvAwcgzFk6hzEkeAHX H6sQ== X-Forwarded-Encrypted: i=1; AJvYcCVIdVVkqLl4K1wcLFm3jDpvt2henoz5u5FdE2TW/HWjmWG3CbM7xCUYlnoa1ORnrwIW97rPXJwnJD95fpDN2ZvWfsmgGTReESZCsr6wsQGfbxm3EZk= X-Gm-Message-State: AOJu0Ywpg6l0FxWuhLM3M7VhEm6+v6RRwQVpnXg6FhIqVBoDxPGGOV5q lH1Sr+FEx6LaLCUI2D0tSbvwnxqVl8p1MAAc8M2Hx0ToPFBgPug1EuJv5ckwBCg= X-Google-Smtp-Source: AGHT+IHa/zXusbJGA4K8cLPLAWn+CnPIA4cHPz+eGb6xCnwX+ovRVMEI/wA60Ad/6sOmnhgR0Mc6ng== X-Received: by 2002:a17:90b:3c50:b0:2a2:19f:dbc7 with SMTP id pm16-20020a17090b3c5000b002a2019fdbc7mr1989065pjb.0.1711697084840; Fri, 29 Mar 2024 00:24:44 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id b8-20020a17090a010800b0029ddac03effsm4971798pjb.11.2024.03.29.00.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Mar 2024 00:24:44 -0700 (PDT) From: Samuel Holland To: Andrew Morton , linux-arm-kernel@lists.infradead.org, x86@kernel.org Cc: linux-kernel@vger.kernel.org, linux-arch@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, Christoph Hellwig , loongarch@lists.linux.dev, amd-gfx@lists.freedesktop.org, Samuel Holland , Borislav Petkov , Catalin Marinas , Dave Hansen , Huacai Chen , Ingo Molnar , Jonathan Corbet , Masahiro Yamada , Nathan Chancellor , Nicolas Schier , Russell King , Thomas Gleixner , Will Deacon , linux-doc@vger.kernel.org, linux-kbuild@vger.kernel.org Subject: [PATCH v4 01/15] arch: Add ARCH_HAS_KERNEL_FPU_SUPPORT Date: Fri, 29 Mar 2024 00:18:16 -0700 Message-ID: <20240329072441.591471-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240329072441.591471-1-samuel.holland@sifive.com> References: <20240329072441.591471-1-samuel.holland@sifive.com> MIME-Version: 1.0 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240329_002446_462578_E3ECAE86 X-CRM114-Status: GOOD ( 30.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Several architectures provide an API to enable the FPU and run floating-point SIMD code in kernel space. However, the function names, header locations, and semantics are inconsistent across architectures, and FPU support may be gated behind other Kconfig options. Provide a standard way for architectures to declare that kernel space FPU support is available. Architectures selecting this option must implement what is currently the most common API (kernel_fpu_begin() and kernel_fpu_end(), plus a new function kernel_fpu_available()) and provide the appropriate CFLAGS for compiling floating-point C code. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Add documentation explaining the built-time and runtime APIs - Add a linux/fpu.h header for generic isolation enforcement Documentation/core-api/floating-point.rst | 78 +++++++++++++++++++++++ Documentation/core-api/index.rst | 1 + Makefile | 5 ++ arch/Kconfig | 6 ++ include/linux/fpu.h | 12 ++++ 5 files changed, 102 insertions(+) create mode 100644 Documentation/core-api/floating-point.rst create mode 100644 include/linux/fpu.h diff --git a/Documentation/core-api/floating-point.rst b/Documentation/core-api/floating-point.rst new file mode 100644 index 000000000000..a8d0d4b05052 --- /dev/null +++ b/Documentation/core-api/floating-point.rst @@ -0,0 +1,78 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +Floating-point API +================== + +Kernel code is normally prohibited from using floating-point (FP) registers or +instructions, including the C float and double data types. This rule reduces +system call overhead, because the kernel does not need to save and restore the +userspace floating-point register state. + +However, occasionally drivers or library functions may need to include FP code. +This is supported by isolating the functions containing FP code to a separate +translation unit (a separate source file), and saving/restoring the FP register +state around calls to those functions. This creates "critical sections" of +floating-point usage. + +The reason for this isolation is to prevent the compiler from generating code +touching the FP registers outside these critical sections. Compilers sometimes +use FP registers to optimize inlined ``memcpy`` or variable assignment, as +floating-point registers may be wider than general-purpose registers. + +Usability of floating-point code within the kernel is architecture-specific. +Additionally, because a single kernel may be configured to support platforms +both with and without a floating-point unit, FPU availability must be checked +both at build time and at run time. + +Several architectures implement the generic kernel floating-point API from +``linux/fpu.h``, as described below. Some other architectures implement their +own unique APIs, which are documented separately. + +Build-time API +-------------- + +Floating-point code may be built if the option ``ARCH_HAS_KERNEL_FPU_SUPPORT`` +is enabled. For C code, such code must be placed in a separate file, and that +file must have its compilation flags adjusted using the following pattern:: + + CFLAGS_foo.o += $(CC_FLAGS_FPU) + CFLAGS_REMOVE_foo.o += $(CC_FLAGS_NO_FPU) + +Architectures are expected to define one or both of these variables in their +top-level Makefile as needed. For example:: + + CC_FLAGS_FPU := -mhard-float + +or:: + + CC_FLAGS_NO_FPU := -msoft-float + +Normal kernel code is assumed to use the equivalent of ``CC_FLAGS_NO_FPU``. + +Runtime API +----------- + +The runtime API is provided in ``linux/fpu.h``. This header cannot be included +from files implementing FP code (those with their compilation flags adjusted as +above). Instead, it must be included when defining the FP critical sections. + +.. c:function:: bool kernel_fpu_available( void ) + + This function reports if floating-point code can be used on this CPU or + platform. The value returned by this function is not expected to change + at runtime, so it only needs to be called once, not before every + critical section. + +.. c:function:: void kernel_fpu_begin( void ) + void kernel_fpu_end( void ) + + These functions create a floating-point critical section. It is only + valid to call ``kernel_fpu_begin()`` after a previous call to + ``kernel_fpu_available()`` returned ``true``. These functions are only + guaranteed to be callable from (preemptible or non-preemptible) process + context. + + Preemption may be disabled inside critical sections, so their size + should be minimized. They are *not* required to be reentrant. If the + caller expects to nest critical sections, it must implement its own + reference counting. diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index 7a3a08d81f11..974beccd671f 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -48,6 +48,7 @@ Library functionality that is used throughout the kernel. errseq wrappers/atomic_t wrappers/atomic_bitops + floating-point Low level entry and exit ======================== diff --git a/Makefile b/Makefile index 763b6792d3d5..710f65e4249d 100644 --- a/Makefile +++ b/Makefile @@ -964,6 +964,11 @@ KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif +# Architectures can define flags to add/remove for floating-point support +CC_FLAGS_FPU += -D_LINUX_FPU_COMPILATION_UNIT +export CC_FLAGS_FPU +export CC_FLAGS_NO_FPU + ifneq ($(CONFIG_FUNCTION_ALIGNMENT),0) # Set the minimal function alignment. Use the newer GCC option # -fmin-function-alignment if it is available, or fall back to -falign-funtions. diff --git a/arch/Kconfig b/arch/Kconfig index 9f066785bb71..8e34b3acf73d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1569,6 +1569,12 @@ config ARCH_HAS_NONLEAF_PMD_YOUNG address translations. Page table walkers that clear the accessed bit may use this capability to reduce their search space. +config ARCH_HAS_KERNEL_FPU_SUPPORT + bool + help + Architectures that select this option can run floating-point code in + the kernel, as described in Documentation/core-api/floating-point.rst. + source "kernel/gcov/Kconfig" source "scripts/gcc-plugins/Kconfig" diff --git a/include/linux/fpu.h b/include/linux/fpu.h new file mode 100644 index 000000000000..2fb63e22913b --- /dev/null +++ b/include/linux/fpu.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _LINUX_FPU_H +#define _LINUX_FPU_H + +#ifdef _LINUX_FPU_COMPILATION_UNIT +#error FP code must be compiled separately. See Documentation/core-api/floating-point.rst. +#endif + +#include + +#endif -- 2.44.0 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.ozlabs.org (lists.ozlabs.org [112.213.38.117]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 1C060C6FD1F for ; Fri, 29 Mar 2024 07:25:36 +0000 (UTC) Authentication-Results: lists.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=sifive.com header.i=@sifive.com header.a=rsa-sha256 header.s=google header.b=exAz6dEw; dkim-atps=neutral Received: from boromir.ozlabs.org (localhost [IPv6:::1]) by lists.ozlabs.org (Postfix) with ESMTP id 4V5X4Z4LDpz3dVH for ; Fri, 29 Mar 2024 18:25:34 +1100 (AEDT) Authentication-Results: lists.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=sifive.com header.i=@sifive.com header.a=rsa-sha256 header.s=google header.b=exAz6dEw; dkim-atps=neutral Authentication-Results: lists.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=sifive.com (client-ip=2607:f8b0:4864:20::102a; helo=mail-pj1-x102a.google.com; envelope-from=samuel.holland@sifive.com; receiver=lists.ozlabs.org) Received: from mail-pj1-x102a.google.com (mail-pj1-x102a.google.com [IPv6:2607:f8b0:4864:20::102a]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 4V5X3j5mGGz3cNt for ; Fri, 29 Mar 2024 18:24:48 +1100 (AEDT) Received: by mail-pj1-x102a.google.com with SMTP id 98e67ed59e1d1-2a20e595351so643817a91.3 for ; Fri, 29 Mar 2024 00:24:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sifive.com; s=google; t=1711697085; x=1712301885; darn=lists.ozlabs.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=exAz6dEw9Shv2BsSovfAy6R3hnp/y06lftsxVbmUxAMe4s63NBcoQ4rh0ae/WffCJg zHeteok4y0ZeKU8calbW+YEw/hMdfZlQcT5CemmTtqQCfSB6bYZngD1t+PAGnHfwobkn ePo0MWLMp4ZQDubY+joSrNNpWXc4GFLx0sDqSEqmOKlgRM3ZzL4MJ5gRal7CNlm9RbRV 2cxdWlf0m9cOK5wCFIuaEihsy120mz3ctq4zgOp6ZR3hYSSGGP4Vdz5VHCjCfBwJ99Pa P67wlKi4dXr6zrj5A6GyvB2RCr9ZyxJVWgxkWrfzybRC6bId4m/St0Jf4faWb6/zksuT qLqg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1711697085; x=1712301885; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=gJgb6TiBTsQDE1YCv6FX1Cn5RHV4TaKIlUyS6ICHf2c=; b=iT9SWRJQRjAxE/1acoLGCZcdSS7p/2cxyuIchT1rjVINpKh9NkZcaY28v/z1RDC5t4 ryjBdElAJtt3UlJfz964jdT1FJPntuxaibs7k+6Y8TRbd8i64krERLEDKYtWghjjH3SM Uko9vfZ7RUpPuyqPDX9kbIs7JNCwo7mLpW6M+eY5HXGFU4R0Epgvb2w3ARLMICl+6Pl+ A4Z5IakLU1+S/6FYfgjUUxbUeQ45h3E2X83R65SVuazDTsJ2sbsJIJ836cj7ZpTU2VDg dWzgBHG0vlo79ycI19I/rgtPsUvQq1ErWijAU8NGgCVo1LV6ZpLtkdUc+Gww0EY82luD F8UA== X-Forwarded-Encrypted: i=1; AJvYcCUD5Pa9xuLWp72eJ49kpeaTqWbWrWVq6F6x/DLeyMjQz5yWvdsQOePwli8fwczr/LTSzydZvjn5tVM2uZEqHO3WjN8SIq9kSlSzx2b8AQ== X-Gm-Message-State: AOJu0YwqXHCeS7s+no0KsMkWFu91ROtIqrruawUQDB/D0otZBChOcxrE +hYpTwUyp58y5p+K+Ckm/L1LPMJMzgAasnpeKTiDBs/rvWY93EaCR5Le0+zapC8= X-Google-Smtp-Source: AGHT+IHa/zXusbJGA4K8cLPLAWn+CnPIA4cHPz+eGb6xCnwX+ovRVMEI/wA60Ad/6sOmnhgR0Mc6ng== X-Received: by 2002:a17:90b:3c50:b0:2a2:19f:dbc7 with SMTP id pm16-20020a17090b3c5000b002a2019fdbc7mr1989065pjb.0.1711697084840; Fri, 29 Mar 2024 00:24:44 -0700 (PDT) Received: from sw06.internal.sifive.com ([4.53.31.132]) by smtp.gmail.com with ESMTPSA id b8-20020a17090a010800b0029ddac03effsm4971798pjb.11.2024.03.29.00.24.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 29 Mar 2024 00:24:44 -0700 (PDT) From: Samuel Holland To: Andrew Morton , linux-arm-kernel@lists.infradead.org, x86@kernel.org Subject: [PATCH v4 01/15] arch: Add ARCH_HAS_KERNEL_FPU_SUPPORT Date: Fri, 29 Mar 2024 00:18:16 -0700 Message-ID: <20240329072441.591471-2-samuel.holland@sifive.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: <20240329072441.591471-1-samuel.holland@sifive.com> References: <20240329072441.591471-1-samuel.holland@sifive.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-BeenThere: linuxppc-dev@lists.ozlabs.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Linux on PowerPC Developers Mail List List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: linux-doc@vger.kernel.org, Catalin Marinas , Dave Hansen , linux-riscv@lists.infradead.org, Will Deacon , Christoph Hellwig , linux-arch@vger.kernel.org, Jonathan Corbet , Masahiro Yamada , Huacai Chen , Russell King , amd-gfx@lists.freedesktop.org, Ingo Molnar , Nicolas Schier , linux-kbuild@vger.kernel.org, Nathan Chancellor , Borislav Petkov , loongarch@lists.linux.dev, Thomas Gleixner , linux-kernel@vger.kernel.org, Samuel Holland , linuxppc-dev@lists.ozlabs.org Errors-To: linuxppc-dev-bounces+linuxppc-dev=archiver.kernel.org@lists.ozlabs.org Sender: "Linuxppc-dev" Several architectures provide an API to enable the FPU and run floating-point SIMD code in kernel space. However, the function names, header locations, and semantics are inconsistent across architectures, and FPU support may be gated behind other Kconfig options. Provide a standard way for architectures to declare that kernel space FPU support is available. Architectures selecting this option must implement what is currently the most common API (kernel_fpu_begin() and kernel_fpu_end(), plus a new function kernel_fpu_available()) and provide the appropriate CFLAGS for compiling floating-point C code. Suggested-by: Christoph Hellwig Reviewed-by: Christoph Hellwig Signed-off-by: Samuel Holland --- (no changes since v2) Changes in v2: - Add documentation explaining the built-time and runtime APIs - Add a linux/fpu.h header for generic isolation enforcement Documentation/core-api/floating-point.rst | 78 +++++++++++++++++++++++ Documentation/core-api/index.rst | 1 + Makefile | 5 ++ arch/Kconfig | 6 ++ include/linux/fpu.h | 12 ++++ 5 files changed, 102 insertions(+) create mode 100644 Documentation/core-api/floating-point.rst create mode 100644 include/linux/fpu.h diff --git a/Documentation/core-api/floating-point.rst b/Documentation/core-api/floating-point.rst new file mode 100644 index 000000000000..a8d0d4b05052 --- /dev/null +++ b/Documentation/core-api/floating-point.rst @@ -0,0 +1,78 @@ +.. SPDX-License-Identifier: GPL-2.0+ + +Floating-point API +================== + +Kernel code is normally prohibited from using floating-point (FP) registers or +instructions, including the C float and double data types. This rule reduces +system call overhead, because the kernel does not need to save and restore the +userspace floating-point register state. + +However, occasionally drivers or library functions may need to include FP code. +This is supported by isolating the functions containing FP code to a separate +translation unit (a separate source file), and saving/restoring the FP register +state around calls to those functions. This creates "critical sections" of +floating-point usage. + +The reason for this isolation is to prevent the compiler from generating code +touching the FP registers outside these critical sections. Compilers sometimes +use FP registers to optimize inlined ``memcpy`` or variable assignment, as +floating-point registers may be wider than general-purpose registers. + +Usability of floating-point code within the kernel is architecture-specific. +Additionally, because a single kernel may be configured to support platforms +both with and without a floating-point unit, FPU availability must be checked +both at build time and at run time. + +Several architectures implement the generic kernel floating-point API from +``linux/fpu.h``, as described below. Some other architectures implement their +own unique APIs, which are documented separately. + +Build-time API +-------------- + +Floating-point code may be built if the option ``ARCH_HAS_KERNEL_FPU_SUPPORT`` +is enabled. For C code, such code must be placed in a separate file, and that +file must have its compilation flags adjusted using the following pattern:: + + CFLAGS_foo.o += $(CC_FLAGS_FPU) + CFLAGS_REMOVE_foo.o += $(CC_FLAGS_NO_FPU) + +Architectures are expected to define one or both of these variables in their +top-level Makefile as needed. For example:: + + CC_FLAGS_FPU := -mhard-float + +or:: + + CC_FLAGS_NO_FPU := -msoft-float + +Normal kernel code is assumed to use the equivalent of ``CC_FLAGS_NO_FPU``. + +Runtime API +----------- + +The runtime API is provided in ``linux/fpu.h``. This header cannot be included +from files implementing FP code (those with their compilation flags adjusted as +above). Instead, it must be included when defining the FP critical sections. + +.. c:function:: bool kernel_fpu_available( void ) + + This function reports if floating-point code can be used on this CPU or + platform. The value returned by this function is not expected to change + at runtime, so it only needs to be called once, not before every + critical section. + +.. c:function:: void kernel_fpu_begin( void ) + void kernel_fpu_end( void ) + + These functions create a floating-point critical section. It is only + valid to call ``kernel_fpu_begin()`` after a previous call to + ``kernel_fpu_available()`` returned ``true``. These functions are only + guaranteed to be callable from (preemptible or non-preemptible) process + context. + + Preemption may be disabled inside critical sections, so their size + should be minimized. They are *not* required to be reentrant. If the + caller expects to nest critical sections, it must implement its own + reference counting. diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst index 7a3a08d81f11..974beccd671f 100644 --- a/Documentation/core-api/index.rst +++ b/Documentation/core-api/index.rst @@ -48,6 +48,7 @@ Library functionality that is used throughout the kernel. errseq wrappers/atomic_t wrappers/atomic_bitops + floating-point Low level entry and exit ======================== diff --git a/Makefile b/Makefile index 763b6792d3d5..710f65e4249d 100644 --- a/Makefile +++ b/Makefile @@ -964,6 +964,11 @@ KBUILD_CFLAGS += $(CC_FLAGS_CFI) export CC_FLAGS_CFI endif +# Architectures can define flags to add/remove for floating-point support +CC_FLAGS_FPU += -D_LINUX_FPU_COMPILATION_UNIT +export CC_FLAGS_FPU +export CC_FLAGS_NO_FPU + ifneq ($(CONFIG_FUNCTION_ALIGNMENT),0) # Set the minimal function alignment. Use the newer GCC option # -fmin-function-alignment if it is available, or fall back to -falign-funtions. diff --git a/arch/Kconfig b/arch/Kconfig index 9f066785bb71..8e34b3acf73d 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -1569,6 +1569,12 @@ config ARCH_HAS_NONLEAF_PMD_YOUNG address translations. Page table walkers that clear the accessed bit may use this capability to reduce their search space. +config ARCH_HAS_KERNEL_FPU_SUPPORT + bool + help + Architectures that select this option can run floating-point code in + the kernel, as described in Documentation/core-api/floating-point.rst. + source "kernel/gcov/Kconfig" source "scripts/gcc-plugins/Kconfig" diff --git a/include/linux/fpu.h b/include/linux/fpu.h new file mode 100644 index 000000000000..2fb63e22913b --- /dev/null +++ b/include/linux/fpu.h @@ -0,0 +1,12 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef _LINUX_FPU_H +#define _LINUX_FPU_H + +#ifdef _LINUX_FPU_COMPILATION_UNIT +#error FP code must be compiled separately. See Documentation/core-api/floating-point.rst. +#endif + +#include + +#endif -- 2.44.0