From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-23.3 required=3.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, INCLUDES_CR_TRAILER,INCLUDES_PATCH,MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS, URIBL_BLOCKED,USER_IN_DEF_DKIM_WL autolearn=unavailable autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id B9ED7C48BE5 for ; Sat, 12 Jun 2021 17:26:28 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by mail.kernel.org (Postfix) with ESMTP id 8ED9D61287 for ; Sat, 12 Jun 2021 17:26:28 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230130AbhFLR20 (ORCPT ); Sat, 12 Jun 2021 13:28:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:38594 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229532AbhFLR2Z (ORCPT ); Sat, 12 Jun 2021 13:28:25 -0400 Received: from mail-ej1-x633.google.com (mail-ej1-x633.google.com [IPv6:2a00:1450:4864:20::633]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 2FE5BC0617AF for ; Sat, 12 Jun 2021 10:26:11 -0700 (PDT) Received: by mail-ej1-x633.google.com with SMTP id k25so9359670eja.9 for ; Sat, 12 Jun 2021 10:26:11 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=K87q3ygNLZInfR0ONZVqvHllcQ29xNPoiT8D9b1JLuw=; b=ZOkHFj6hSeWLUzU5sviAC8WsxV93dSuuwZ93byQQ+gjSihsf73nYF355no0qktkSba NoyiypLwIjK48xpyeDXoZn3i3Scdlr3KC+NB+jfAm/F021o+wt/xGmLWO82eqGCyBOSf LVIwxd8piswNkxPOnbivae67ddppYOLaVXMBYY5sR92J5bMSqqpHLZyD3QoOIH2yKD10 mMqBlzigrfqANNEEoyCkfjui49nI48BnVqcEhk44QYDLnhPmLTmsnkUV25aDxOzSAzGJ ZSzQ9Sm8hk6NtEqRAgFWIGOlNvSzgMnln5ALbLw0EPaHHKGikF1PIi3bMqyQtx8vBxM2 ohmg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=K87q3ygNLZInfR0ONZVqvHllcQ29xNPoiT8D9b1JLuw=; b=nYWz6p7pyVZE2Pp3JZ8CPRIH4tIRgVZmiGetOhjAmvxuD1FYlbo5SgePvMOLANB9bA N4yd72ERSwe+bBewoiW6YCElyVxs8a03njcY7rQvKuVKJpYoJAVPA6OCZtC9eZr7cA7b x7j9YHbR6dggaFpEnJW5AUTeKXEug7PbP2Stf3/GnbVjj9MA09QLc/PaZpLX/dbX/Rok cLTkshjHOa5nx3F6mPBj19MwGWOx81exBXP6+0ne/X1BmubeXp2kEHt5TY27iEcyQoee FYMa2wZ4L8OYOtsgOlqIKRjUKoSmj5woStsjj8gkN74dySQiwmUoWCc7bK7QWX+cgZRu Lt8Q== X-Gm-Message-State: AOAM53095pVgmd521LHuiB3pQJrdTZVNDI0MCGNQs5oyCG7f89C3kT6c hF+pHEdZHEauZRHA19ULenK/X8q2SwgkWIcQSDe/ X-Google-Smtp-Source: ABdhPJzcY3PJ0i0kRZsXmsHSJkM2+xDm70FOyUiKPcUcD26i/kuZ5RA6kAtb5kTg/ogkCIII1l69j+DJn/YKYIB/yzQ= X-Received: by 2002:a17:906:7188:: with SMTP id h8mr8574666ejk.529.1623518769255; Sat, 12 Jun 2021 10:26:09 -0700 (PDT) MIME-Version: 1.0 References: <20210111081821.3041587-1-morbo@google.com> <20210407211704.367039-1-morbo@google.com> In-Reply-To: From: Bill Wendling Date: Sat, 12 Jun 2021 10:25:57 -0700 Message-ID: Subject: Re: [PATCH v9] pgo: add clang's Profile Guided Optimization infrastructure To: Peter Zijlstra Cc: Kees Cook , Jonathan Corbet , Masahiro Yamada , Linux Doc Mailing List , LKML , Linux Kbuild mailing list , clang-built-linux , Andrew Morton , Nathan Chancellor , Nick Desaulniers , Sami Tolvanen , Fangrui Song , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sat, Jun 12, 2021 at 9:59 AM Peter Zijlstra wrote: > > On Wed, Apr 07, 2021 at 02:17:04PM -0700, Bill Wendling wrote: > > From: Sami Tolvanen > > > > Enable the use of clang's Profile-Guided Optimization[1]. To generate a > > profile, the kernel is instrumented with PGO counters, a representative > > workload is run, and the raw profile data is collected from > > /sys/kernel/debug/pgo/profraw. > > > > The raw profile data must be processed by clang's "llvm-profdata" tool > > before it can be used during recompilation: > > > > $ cp /sys/kernel/debug/pgo/profraw vmlinux.profraw > > $ llvm-profdata merge --output=vmlinux.profdata vmlinux.profraw > > > > Multiple raw profiles may be merged during this step. > > > > The data can now be used by the compiler: > > > > $ make LLVM=1 KCFLAGS=-fprofile-use=vmlinux.profdata ... > > > > This initial submission is restricted to x86, as that's the platform we > > know works. This restriction can be lifted once other platforms have > > been verified to work with PGO. > > *sigh*, and not a single x86 person on Cc, how nice :-/ > This tool is generic and, despite the fact that it's first enabled for x86, it contains no x86-specific code. The reason we're restricting it to x86 is because it's the platform we tested on. > > Note that this method of profiling the kernel is clang-native, unlike > > the clang support in kernel/gcov. > > > > [1] https://clang.llvm.org/docs/UsersManual.html#profile-guided-optimization > > Also, and I don't see this answered *anywhere*, why are you not using > perf for this? Your link even mentions Sampling Profilers (and I happen > to know there's been significant effort to make perf output work as > input for the PGO passes of the various compilers). > Instruction-based (non-sampling) profiling gives us a better context-sensitive profile, making PGO more impactful. It's also useful for coverage whereas sampling profiles cannot. > > Signed-off-by: Sami Tolvanen > > Co-developed-by: Bill Wendling > > Signed-off-by: Bill Wendling > > Tested-by: Nick Desaulniers > > Reviewed-by: Nick Desaulniers > > Reviewed-by: Fangrui Song > > --- > > Documentation/dev-tools/index.rst | 1 + > > Documentation/dev-tools/pgo.rst | 127 +++++++++ > > MAINTAINERS | 9 + > > Makefile | 3 + > > arch/Kconfig | 1 + > > arch/x86/Kconfig | 1 + > > arch/x86/boot/Makefile | 1 + > > arch/x86/boot/compressed/Makefile | 1 + > > arch/x86/crypto/Makefile | 4 + > > arch/x86/entry/vdso/Makefile | 1 + > > arch/x86/kernel/vmlinux.lds.S | 2 + > > arch/x86/platform/efi/Makefile | 1 + > > arch/x86/purgatory/Makefile | 1 + > > arch/x86/realmode/rm/Makefile | 1 + > > arch/x86/um/vdso/Makefile | 1 + > > drivers/firmware/efi/libstub/Makefile | 1 + > > include/asm-generic/vmlinux.lds.h | 34 +++ > > kernel/Makefile | 1 + > > kernel/pgo/Kconfig | 35 +++ > > kernel/pgo/Makefile | 5 + > > kernel/pgo/fs.c | 389 ++++++++++++++++++++++++++ > > kernel/pgo/instrument.c | 189 +++++++++++++ > > kernel/pgo/pgo.h | 203 ++++++++++++++ > > scripts/Makefile.lib | 10 + > > 24 files changed, 1022 insertions(+) > > create mode 100644 Documentation/dev-tools/pgo.rst > > create mode 100644 kernel/pgo/Kconfig > > create mode 100644 kernel/pgo/Makefile > > create mode 100644 kernel/pgo/fs.c > > create mode 100644 kernel/pgo/instrument.c > > create mode 100644 kernel/pgo/pgo.h > > > --- a/Makefile > > +++ b/Makefile > > @@ -660,6 +660,9 @@ endif # KBUILD_EXTMOD > > # Defaults to vmlinux, but the arch makefile usually adds further targets > > all: vmlinux > > > > +CFLAGS_PGO_CLANG := -fprofile-generate > > +export CFLAGS_PGO_CLANG > > + > > CFLAGS_GCOV := -fprofile-arcs -ftest-coverage \ > > $(call cc-option,-fno-tree-loop-im) \ > > $(call cc-disable-warning,maybe-uninitialized,) > > And which of the many flags in noinstr disables this? > These flags aren't used with PGO. So there's no need to disable them. > Basically I would like to NAK this whole thing until someone can > adequately explain the interaction with noinstr and why we need those > many lines of kernel code and can't simply use perf for this. -bw