From: Vitaly Chikunov <vt@altlinux.org>
To: Arnaldo Carvalho de Melo <acme@kernel.org>
Cc: Yonghong Song <yhs@fb.com>,
Andrii Nakryiko <andrii.nakryiko@gmail.com>,
Jiri Olsa <jolsa@kernel.org>, Ian Rogers <irogers@google.com>,
dwarves@vger.kernel.org, bpf@vger.kernel.org,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: pahole v1.24: FAILED: load BTF from vmlinux: Invalid argument
Date: Sun, 28 Aug 2022 04:41:59 +0300 [thread overview]
Message-ID: <20220828014159.x2u4esk2ijkjlitv@altlinux.org> (raw)
In-Reply-To: <YwpjkbMkgDIufjtO@kernel.org>
On Sat, Aug 27, 2022 at 03:33:53PM -0300, Arnaldo Carvalho de Melo wrote:
> Em Sat, Aug 27, 2022 at 08:33:10PM +0300, Vitaly Chikunov escreveu:
> > On Fri, Aug 26, 2022 at 01:37:24PM -0300, Arnaldo Carvalho de Melo wrote:
> > > Em Fri, Aug 26, 2022 at 03:52:12PM +0200, Jiri Olsa escreveu:
> > > > On Fri, Aug 26, 2022 at 05:59:44AM +0300, Vitaly Chikunov wrote:
> > > > > > [ 0.767649] Run /init as init process
> > > > > > [ 0.770858] BPF:[593] ENUM perf_event_task_context
> > > > > > [ 0.771262] BPF:size=4 vlen=4
> > > > > > [ 0.771511] BPF:
> > > > > > [ 0.771680] BPF:Invalid btf_info kind_flag
> > > > > > [ 0.772016] BPF:
>
> > > > I can see the same on 5.15, it looks like the libbpf change that
> > > > pahole is compiled with is setting the type's kflag for values < 0:
> > > > (which is the case for perf_event_task_context enum first value)
>
> > > > dffbbdc2d988 libbpf: Add enum64 parsing and new enum64 public API
>
> > > > but IIUC kflag should stay zero for normal enum otherwise the btf meta
> > > > verifier screams
>
> > > > if I compile pahole with the libbpf change below I can boot 5.15 kernel
> > > > normally
>
> > > > Yonghong, any idea?
>
> > > This made me try to build pahole with the system libbpf instead of with
> > > the one that goes with it, here, testing with libbpf 0.7.0 it wasn't
> > > building as BTF_KIND_ENUM64 came with libbpf 1.0 so I added the
> > > following patch to again allow with the system libbpf, i.e. using:
>
> > > $ cmake -DCMAKE_BUILD_TYPE=Release -DLIBBPF_EMBEDDED=Off
>
> > > This will return errors if trying to encode or load enum64 tags, but
> > > disabling it, as done with kernels not supporting BTF_KIND_ENUM64 should
> > > now work, can you please test and report results?
>
> > > Vitaly I checked and alt:p9 has libbpf 0.2, which is really old, unsure
> > > if it would build there, but alt:sisyphus has 0.8.0, so should work
> > > there, please try.
>
> > Perhaps this does not need to be tested since Jiri found the solution?
>
> Right, what I suggested was just an alternative, to use the system
> libbpf package instead of the one that ships with pahole, as the older
> libbpf woudln't have the problem hypothesized by Jiri.
>
> But it has already been determined that the problem is elsewhere, so
> this just was useful to make me realize building with libbpf-devel was
> broken due to BTF_KIND_ENUM64, which should be fixed with this patch.
>
> > BTW, alt:p9 is older ALT branch only for security updates, current
> > branch is alt:p10, so it's relevant to replace p9 testing with p10, for
> > new features.
>
> Sure, I go on adding containers and drop them when it is not possible to
> build with them, so far this is what I have for alt:
>
> 9 25.26 alt:p8 : Ok x86_64-alt-linux-gcc (GCC) 5.3.1 20151207 (ALT p8 5.3.1-alt3.M80P.1)
> 10 81.96 alt:p9 : Ok x86_64-alt-linux-gcc (GCC) 8.4.1 20200305 (ALT p9 8.4.1-alt0.p9.1) , clang version 10.0.0
> 11 94.60 alt:p10 : Ok x86_64-alt-linux-gcc (GCC) 10.3.1 20210703 (ALT Sisyphus 10.3.1-alt2) , clang version 11.0.1
> 12 77.53 alt:sisyphus : Ok x86_64-alt-linux-gcc (GCC) 12.1.1 20220518 (ALT Sisyphus 12.1.1-alt1) , ALT Linux Team clang version 13.0.1
Btw, we also have clang14.0 in sisyphus, it's just non default yet.
>
> p8's clang is too old, disabled, p9 also has:
>
> ENV NO_BUILD_BPF_SKEL=1
>
> clang also too old to build BPF skels, but still good to build perf
> proper.
>
> > I was building dwarves with -DLIBBPF_EMBEDDED=ON, thinking it would be
> > more stable (since they updated not independently from each other), is
> > it recommended to turn in OFF?
>
> What most distros are doing is to ship a libbpf and they expect and work
> to make tools use it, which isn't always possible, like the patch I sent
> shows.
>
> Its a balance on having the latest and greatest libbpf that is needed
> for some feature pahole uses and the desire to not link anything
> statically due to security concerns (having to fix all things linking
> statically with a faulty library).
>
> > ps. I think something needs to be changed somewhere (in kernel src?) or
> > more users will report build failures when switching to the new pahole.
>
> Right, I voiced concerns already about having enum64 encoded by default,
> but its not so simple to solve.
>
> People living on the bleeding edge should not have problems, as recent
> kernels support the new BTF encodings, its just people building older
> kernels with new pahole that will have problems, right?
Yes, but even in Sisyphus (where we try to build bleeding edge stuff) we
build latest stable and longterm kernels, that's why we happen to build
v5.18.19 and v5.15.63 with dwarves 1.24. And 5.15.63 is released just two
days ago so this is not that old.
> By test building against alt:p8 you see that I care about such version
> mismatches, but its a burden, really, to try and cover all cases.
I very much appreciate your work.
Thanks!
>
> - Arnaldo
next prev parent reply other threads:[~2022-08-28 1:42 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-08-25 16:35 pahole v1.24: FAILED: load BTF from vmlinux: Invalid argument Vitaly Chikunov
[not found] ` <CA+JHD904e2TPpz1ybsaaqD+qMTDcueXu4nVcmotEPhxNfGN+Gw@mail.gmail.com>
2022-08-25 17:16 ` Vitaly Chikunov
2022-08-26 2:52 ` Vitaly Chikunov
2022-08-26 2:59 ` Vitaly Chikunov
2022-08-26 13:52 ` Jiri Olsa
2022-08-26 16:37 ` Arnaldo Carvalho de Melo
2022-08-27 17:33 ` Vitaly Chikunov
2022-08-27 18:33 ` Arnaldo Carvalho de Melo
2022-08-28 1:41 ` Vitaly Chikunov [this message]
2022-08-26 16:51 ` Yonghong Song
2022-08-26 17:01 ` Jiri Olsa
2022-08-26 18:53 ` Yonghong Song
2022-08-26 20:19 ` Jiri Olsa
2022-08-27 1:06 ` Vitaly Chikunov
2022-08-26 16:47 ` Yonghong Song
2022-08-26 16:52 ` Arnaldo Carvalho de Melo
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220828014159.x2u4esk2ijkjlitv@altlinux.org \
--to=vt@altlinux.org \
--cc=acme@kernel.org \
--cc=andrii.nakryiko@gmail.com \
--cc=bpf@vger.kernel.org \
--cc=dwarves@vger.kernel.org \
--cc=irogers@google.com \
--cc=jolsa@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).