* [RFC] perf: probe_finder: continue if atleast one probe point found
@ 2016-02-28 11:14 Joel Fernandes
2016-02-28 11:35 ` Taeung Song
0 siblings, 1 reply; 7+ messages in thread
From: Joel Fernandes @ 2016-02-28 11:14 UTC (permalink / raw
To: linux-perf-users, linux-kernel; +Cc: joel.opensrc, agnel.joel
Sometimes for inline functions, perf probe can fail such as if an arguments are
requested at probe points. This is probably because when the compiler inlines,
for some instances it optimizes away arguments. Either way, the DWARF has
missing arguments for certain probe points of inline functions causing 'perf
probe' to fail. I noticed this when probing the C library that ships with my
distribution. With the following patch I am successfully able to record kprobe
events with arguments.
Test
----
./perf probe \
-k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc bytes' -v
Without the patch
-----------------
Matched function: __libc_malloc
found inline addr: 0x831a6
Probe point found: malloc_atfork+150
Searching 'bytes' variable in context.
Converting variable bytes into trace event.
bytes type is long unsigned int.
found inline addr: 0x844a0
Probe point found: __libc_malloc+0
Searching 'bytes' variable in context.
Converting variable bytes into trace event.
bytes type is long unsigned int.
found inline addr: 0x8463b
Probe point found: __malloc_get_state+11
Searching 'bytes' variable in context.
Failed to find 'bytes' in this function.
An error occurred in debuginfo analysis (-2).
Error: Failed to add events. Reason: No such file or directory (Code: -2)
With the patch
--------------
Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
Try to find probe point from debuginfo.
Symbol malloc address found : 844a0
Matched function: __libc_malloc
found inline addr: 0x831a6
Probe point found: malloc_atfork+150
Searching 'bytes' variable in context.
Converting variable bytes into trace event.
bytes type is long unsigned int.
found inline addr: 0x844a0
Probe point found: __libc_malloc+0
Searching 'bytes' variable in context.
Converting variable bytes into trace event.
bytes type is long unsigned int.
found inline addr: 0x8463b
Probe point found: __malloc_get_state+11
Searching 'bytes' variable in context.
Failed to find 'bytes' in this function.
Probe point error, ignoring. Atleast one probe point found
found inline addr: 0x84bba
Probe point found: __libc_realloc+410
Searching 'bytes' variable in context.
Failed to find 'bytes' in this function.
Probe point error, ignoring. Atleast one probe point found
found inline addr: 0x84d08
...
Probe point error, ignoring. Atleast one probe point found
Found 3 probe_trace_events.
Opening /sys/kernel/debug/tracing//uprobe_events write=1
Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6 bytes=%bp:u64
Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0 bytes=%di:u64
Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7 bytes=%di:u64
Added new events:
probe_libc:malloc (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so with bytes)
probe_libc:malloc_1 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so with bytes)
probe_libc:malloc_2 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so with bytes)
Signed-off-by: Joel Fernandes <agnel.joel@gmail.com>
---
tools/perf/util/probe-finder.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index 4ce5c5e..3ac9481 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -1255,6 +1255,10 @@ end:
if (ret) {
clear_probe_trace_event(tev);
tf->ntevs--;
+ if (tf->ntevs != 0) {
+ pr_debug("Ignoring error as atleast one probe point found.\n");
+ ret = 0;
+ }
}
free(args);
return ret;
--
2.5.0
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [RFC] perf: probe_finder: continue if atleast one probe point found
2016-02-28 11:14 [RFC] perf: probe_finder: continue if atleast one probe point found Joel Fernandes
@ 2016-02-28 11:35 ` Taeung Song
[not found] ` <CAD=GYpZtDb3ugJRFQg1karJ_Fb7RsGPpSu0ZbL8i9Oy8x5eZ9A@mail.gmail.com>
0 siblings, 1 reply; 7+ messages in thread
From: Taeung Song @ 2016-02-28 11:35 UTC (permalink / raw
To: Joel Fernandes, linux-perf-users, linux-kernel; +Cc: agnel.joel
Hi, Joel
perf subsystem's maintainers are
Arnaldo Carvalho de Melo <acme@kernel.org>
Peter Zijlstra <peterz@infradead.org>
Ingo Molnar <mingo@redhat.com>
(or Namhyung Kim <namhyung@kernel.org>, Jiri Olsa <jolsa@kernel.org>)
AFAIK,
When sending some patches, it would be better to add them into To: or Cc:
Thanks,
Taeung
On 02/28/2016 08:14 PM, Joel Fernandes wrote:
> Sometimes for inline functions, perf probe can fail such as if an arguments are
> requested at probe points. This is probably because when the compiler inlines,
> for some instances it optimizes away arguments. Either way, the DWARF has
> missing arguments for certain probe points of inline functions causing 'perf
> probe' to fail. I noticed this when probing the C library that ships with my
> distribution. With the following patch I am successfully able to record kprobe
> events with arguments.
>
> Test
> ----
> ./perf probe \
> -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc bytes' -v
>
> Without the patch
> -----------------
> Matched function: __libc_malloc
> found inline addr: 0x831a6
> Probe point found: malloc_atfork+150
> Searching 'bytes' variable in context.
> Converting variable bytes into trace event.
> bytes type is long unsigned int.
> found inline addr: 0x844a0
> Probe point found: __libc_malloc+0
> Searching 'bytes' variable in context.
> Converting variable bytes into trace event.
> bytes type is long unsigned int.
> found inline addr: 0x8463b
> Probe point found: __malloc_get_state+11
> Searching 'bytes' variable in context.
> Failed to find 'bytes' in this function.
> An error occurred in debuginfo analysis (-2).
> Error: Failed to add events. Reason: No such file or directory (Code: -2)
>
> With the patch
> --------------
> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
> Try to find probe point from debuginfo.
> Symbol malloc address found : 844a0
> Matched function: __libc_malloc
> found inline addr: 0x831a6
> Probe point found: malloc_atfork+150
> Searching 'bytes' variable in context.
> Converting variable bytes into trace event.
> bytes type is long unsigned int.
> found inline addr: 0x844a0
> Probe point found: __libc_malloc+0
> Searching 'bytes' variable in context.
> Converting variable bytes into trace event.
> bytes type is long unsigned int.
> found inline addr: 0x8463b
> Probe point found: __malloc_get_state+11
> Searching 'bytes' variable in context.
> Failed to find 'bytes' in this function.
> Probe point error, ignoring. Atleast one probe point found
> found inline addr: 0x84bba
> Probe point found: __libc_realloc+410
> Searching 'bytes' variable in context.
> Failed to find 'bytes' in this function.
> Probe point error, ignoring. Atleast one probe point found
> found inline addr: 0x84d08
> ...
> Probe point error, ignoring. Atleast one probe point found
> Found 3 probe_trace_events.
> Opening /sys/kernel/debug/tracing//uprobe_events write=1
> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6 bytes=%bp:u64
> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0 bytes=%di:u64
> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7 bytes=%di:u64
> Added new events:
> probe_libc:malloc (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so with bytes)
> probe_libc:malloc_1 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so with bytes)
> probe_libc:malloc_2 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so with bytes)
>
> Signed-off-by: Joel Fernandes <agnel.joel@gmail.com>
> ---
> tools/perf/util/probe-finder.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
> index 4ce5c5e..3ac9481 100644
> --- a/tools/perf/util/probe-finder.c
> +++ b/tools/perf/util/probe-finder.c
> @@ -1255,6 +1255,10 @@ end:
> if (ret) {
> clear_probe_trace_event(tev);
> tf->ntevs--;
> + if (tf->ntevs != 0) {
> + pr_debug("Ignoring error as atleast one probe point found.\n");
> + ret = 0;
> + }
> }
> free(args);
> return ret;
>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC] perf: probe_finder: continue if atleast one probe point found
[not found] ` <CAD=GYpZtDb3ugJRFQg1karJ_Fb7RsGPpSu0ZbL8i9Oy8x5eZ9A@mail.gmail.com>
@ 2016-02-29 14:19 ` Arnaldo Carvalho de Melo
2016-03-01 3:09 ` 平松雅巳 / HIRAMATU，MASAMI
0 siblings, 1 reply; 7+ messages in thread
From: Arnaldo Carvalho de Melo @ 2016-02-29 14:19 UTC (permalink / raw
To: Masami Hiramatsu
Cc: Joel Fernandes, Taeung Song, Joel Fernandes, linux-perf-users,
Linux Kernel Mailing List, Peter Zijlstra, Ingo Molnar,
Namhyung Kim, Jiri Olsa
Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
> Taeung,
>
> Thanks for that. As this is an RFC, I have added them now to this email
> thread and when I post the final patch, I'll include them in CC.
>
> All,
> Please provide your comments on my patch.
Masami, are you ok with this patch?
- Arnaldo
> Thanks,
> Joel
>
> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@gmail.com>
> wrote:
>
> > Hi, Joel
> >
> > perf subsystem's maintainers are
> > Arnaldo Carvalho de Melo <acme@kernel.org>
> > Peter Zijlstra <peterz@infradead.org>
> > Ingo Molnar <mingo@redhat.com>
> >
> > (or Namhyung Kim <namhyung@kernel.org>, Jiri Olsa <jolsa@kernel.org>)
> >
> > AFAIK,
> > When sending some patches, it would be better to add them into To: or Cc:
> >
> >
> > Thanks,
> > Taeung
> >
> >
> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
> >
> >> Sometimes for inline functions, perf probe can fail such as if an
> >> arguments are
> >> requested at probe points. This is probably because when the compiler
> >> inlines,
> >> for some instances it optimizes away arguments. Either way, the DWARF has
> >> missing arguments for certain probe points of inline functions causing
> >> 'perf
> >> probe' to fail. I noticed this when probing the C library that ships with
> >> my
> >> distribution. With the following patch I am successfully able to record
> >> kprobe
> >> events with arguments.
> >>
> >> Test
> >> ----
> >> ./perf probe \
> >> -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
> >> bytes' -v
> >>
> >> Without the patch
> >> -----------------
> >> Matched function: __libc_malloc
> >> found inline addr: 0x831a6
> >> Probe point found: malloc_atfork+150
> >> Searching 'bytes' variable in context.
> >> Converting variable bytes into trace event.
> >> bytes type is long unsigned int.
> >> found inline addr: 0x844a0
> >> Probe point found: __libc_malloc+0
> >> Searching 'bytes' variable in context.
> >> Converting variable bytes into trace event.
> >> bytes type is long unsigned int.
> >> found inline addr: 0x8463b
> >> Probe point found: __malloc_get_state+11
> >> Searching 'bytes' variable in context.
> >> Failed to find 'bytes' in this function.
> >> An error occurred in debuginfo analysis (-2).
> >> Error: Failed to add events. Reason: No such file or directory (Code:
> >> -2)
> >>
> >> With the patch
> >> --------------
> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
> >> Try to find probe point from debuginfo.
> >> Symbol malloc address found : 844a0
> >> Matched function: __libc_malloc
> >> found inline addr: 0x831a6
> >> Probe point found: malloc_atfork+150
> >> Searching 'bytes' variable in context.
> >> Converting variable bytes into trace event.
> >> bytes type is long unsigned int.
> >> found inline addr: 0x844a0
> >> Probe point found: __libc_malloc+0
> >> Searching 'bytes' variable in context.
> >> Converting variable bytes into trace event.
> >> bytes type is long unsigned int.
> >> found inline addr: 0x8463b
> >> Probe point found: __malloc_get_state+11
> >> Searching 'bytes' variable in context.
> >> Failed to find 'bytes' in this function.
> >> Probe point error, ignoring. Atleast one probe point found
> >> found inline addr: 0x84bba
> >> Probe point found: __libc_realloc+410
> >> Searching 'bytes' variable in context.
> >> Failed to find 'bytes' in this function.
> >> Probe point error, ignoring. Atleast one probe point found
> >> found inline addr: 0x84d08
> >> ...
> >> Probe point error, ignoring. Atleast one probe point found
> >> Found 3 probe_trace_events.
> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
> >> bytes=%bp:u64
> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
> >> bytes=%di:u64
> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
> >> bytes=%di:u64
> >> Added new events:
> >> probe_libc:malloc (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
> >> with bytes)
> >> probe_libc:malloc_1 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
> >> with bytes)
> >> probe_libc:malloc_2 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
> >> with bytes)
> >>
> >> Signed-off-by: Joel Fernandes <agnel.joel@gmail.com>
> >> ---
> >> tools/perf/util/probe-finder.c | 4 ++++
> >> 1 file changed, 4 insertions(+)
> >>
> >> diff --git a/tools/perf/util/probe-finder.c
> >> b/tools/perf/util/probe-finder.c
> >> index 4ce5c5e..3ac9481 100644
> >> --- a/tools/perf/util/probe-finder.c
> >> +++ b/tools/perf/util/probe-finder.c
> >> @@ -1255,6 +1255,10 @@ end:
> >> if (ret) {
> >> clear_probe_trace_event(tev);
> >> tf->ntevs--;
> >> + if (tf->ntevs != 0) {
> >> + pr_debug("Ignoring error as atleast one probe
> >> point found.\n");
> >> + ret = 0;
> >> + }
> >> }
> >> free(args);
> >> return ret;
> >>
> >>
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: [RFC] perf: probe_finder: continue if atleast one probe point found
2016-02-29 14:19 ` Arnaldo Carvalho de Melo
@ 2016-03-01 3:09 ` 平松雅巳 / HIRAMATU，MASAMI
2016-03-01 4:29 ` 平松雅巳 / HIRAMATU，MASAMI
0 siblings, 1 reply; 7+ messages in thread
From: 平松雅巳 / HIRAMATU，MASAMI @ 2016-03-01 3:09 UTC (permalink / raw
To: 'Arnaldo Carvalho de Melo'
Cc: Joel Fernandes, Taeung Song, Joel Fernandes,
linux-perf-users@vger.kernel.org, Linux Kernel Mailing List,
Peter Zijlstra, Ingo Molnar, Namhyung Kim, Jiri Olsa,
sysp-manager
Hi,
>From: Arnaldo Carvalho de Melo [mailto:acme@kernel.org]
>
>Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
>> Taeung,
>>
>> Thanks for that. As this is an RFC, I have added them now to this email
>> thread and when I post the final patch, I'll include them in CC.
>>
>> All,
>> Please provide your comments on my patch.
>
>Masami, are you ok with this patch?
Hmm, I think this is not enough. I concider that the case if users
expect probing on "__malloc_get_state+11", just ignoring it may
confuse or mislead them that perf-probe has a bug not to find it.
IOW, this was by design that perf probe refuses all probe point
if it hits any error.
I think we have 2 options, one is adding an option (or reuse --force)
to ignore error on finding probe points (but this MUST show the error
message by pr_error.) The other is extending perf probe's argument
format so that it can accept local-vars which may not exist, e.g.
'malloc *bytes' or 'malloc {bytes,}' :)
Thank you,
>
>- Arnaldo
>
>> Thanks,
>> Joel
>>
>> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@gmail.com>
>> wrote:
>>
>> > Hi, Joel
>> >
>> > perf subsystem's maintainers are
>> > Arnaldo Carvalho de Melo <acme@kernel.org>
>> > Peter Zijlstra <peterz@infradead.org>
>> > Ingo Molnar <mingo@redhat.com>
>> >
>> > (or Namhyung Kim <namhyung@kernel.org>, Jiri Olsa <jolsa@kernel.org>)
>> >
>> > AFAIK,
>> > When sending some patches, it would be better to add them into To: or Cc:
>> >
>> >
>> > Thanks,
>> > Taeung
>> >
>> >
>> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
>> >
>> >> Sometimes for inline functions, perf probe can fail such as if an
>> >> arguments are
>> >> requested at probe points. This is probably because when the compiler
>> >> inlines,
>> >> for some instances it optimizes away arguments. Either way, the DWARF has
>> >> missing arguments for certain probe points of inline functions causing
>> >> 'perf
>> >> probe' to fail. I noticed this when probing the C library that ships with
>> >> my
>> >> distribution. With the following patch I am successfully able to record
>> >> kprobe
>> >> events with arguments.
>> >>
>> >> Test
>> >> ----
>> >> ./perf probe \
>> >> -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
>> >> bytes' -v
>> >>
>> >> Without the patch
>> >> -----------------
>> >> Matched function: __libc_malloc
>> >> found inline addr: 0x831a6
>> >> Probe point found: malloc_atfork+150
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x844a0
>> >> Probe point found: __libc_malloc+0
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x8463b
>> >> Probe point found: __malloc_get_state+11
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> An error occurred in debuginfo analysis (-2).
>> >> Error: Failed to add events. Reason: No such file or directory (Code:
>> >> -2)
>> >>
>> >> With the patch
>> >> --------------
>> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
>> >> Try to find probe point from debuginfo.
>> >> Symbol malloc address found : 844a0
>> >> Matched function: __libc_malloc
>> >> found inline addr: 0x831a6
>> >> Probe point found: malloc_atfork+150
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x844a0
>> >> Probe point found: __libc_malloc+0
>> >> Searching 'bytes' variable in context.
>> >> Converting variable bytes into trace event.
>> >> bytes type is long unsigned int.
>> >> found inline addr: 0x8463b
>> >> Probe point found: __malloc_get_state+11
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> found inline addr: 0x84bba
>> >> Probe point found: __libc_realloc+410
>> >> Searching 'bytes' variable in context.
>> >> Failed to find 'bytes' in this function.
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> found inline addr: 0x84d08
>> >> ...
>> >> Probe point error, ignoring. Atleast one probe point found
>> >> Found 3 probe_trace_events.
>> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
>> >> bytes=%bp:u64
>> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
>> >> bytes=%di:u64
>> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
>> >> bytes=%di:u64
>> >> Added new events:
>> >> probe_libc:malloc (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >> probe_libc:malloc_1 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >> probe_libc:malloc_2 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>> >> with bytes)
>> >>
>> >> Signed-off-by: Joel Fernandes <agnel.joel@gmail.com>
>> >> ---
>> >> tools/perf/util/probe-finder.c | 4 ++++
>> >> 1 file changed, 4 insertions(+)
>> >>
>> >> diff --git a/tools/perf/util/probe-finder.c
>> >> b/tools/perf/util/probe-finder.c
>> >> index 4ce5c5e..3ac9481 100644
>> >> --- a/tools/perf/util/probe-finder.c
>> >> +++ b/tools/perf/util/probe-finder.c
>> >> @@ -1255,6 +1255,10 @@ end:
>> >> if (ret) {
>> >> clear_probe_trace_event(tev);
>> >> tf->ntevs--;
>> >> + if (tf->ntevs != 0) {
>> >> + pr_debug("Ignoring error as atleast one probe
>> >> point found.\n");
>> >> + ret = 0;
>> >> + }
>> >> }
>> >> free(args);
>> >> return ret;
>> >>
>> >>
^ permalink raw reply [flat|nested] 7+ messages in thread
* RE: [RFC] perf: probe_finder: continue if atleast one probe point found
2016-03-01 3:09 ` 平松雅巳 / HIRAMATU，MASAMI
@ 2016-03-01 4:29 ` 平松雅巳 / HIRAMATU，MASAMI
2016-03-01 10:34 ` Joel Fernandes
0 siblings, 1 reply; 7+ messages in thread
From: 平松雅巳 / HIRAMATU，MASAMI @ 2016-03-01 4:29 UTC (permalink / raw
To: 'ltc-kernel@ml.yrl.intra.hitachi.co.jp',
'Arnaldo Carvalho de Melo'
Cc: Joel Fernandes, Taeung Song, Joel Fernandes,
linux-perf-users@vger.kernel.org, Linux Kernel Mailing List,
Peter Zijlstra, Ingo Molnar, Namhyung Kim, Jiri Olsa,
sysp-manager
BTW,
./perf probe \
-k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc $params'
might help your case. $params is expanded to function parameters automatically
and if there is no parameters, it is just ignored :)
Thank you,
>From: 平松雅巳 / HIRAMATU，MASAMI [mailto:masami.hiramatsu.pt@hitachi.com]
>
>Hi,
>
>>From: Arnaldo Carvalho de Melo [mailto:acme@kernel.org]
>>
>>Em Sun, Feb 28, 2016 at 03:49:44AM -0800, Joel Fernandes escreveu:
>>> Taeung,
>>>
>>> Thanks for that. As this is an RFC, I have added them now to this email
>>> thread and when I post the final patch, I'll include them in CC.
>>>
>>> All,
>>> Please provide your comments on my patch.
>>
>>Masami, are you ok with this patch?
>
>Hmm, I think this is not enough. I concider that the case if users
>expect probing on "__malloc_get_state+11", just ignoring it may
>confuse or mislead them that perf-probe has a bug not to find it.
>
>IOW, this was by design that perf probe refuses all probe point
>if it hits any error.
>
>I think we have 2 options, one is adding an option (or reuse --force)
>to ignore error on finding probe points (but this MUST show the error
>message by pr_error.) The other is extending perf probe's argument
>format so that it can accept local-vars which may not exist, e.g.
>'malloc *bytes' or 'malloc {bytes,}' :)
>
>Thank you,
>
>
>>
>>- Arnaldo
>>
>>> Thanks,
>>> Joel
>>>
>>> On Sun, Feb 28, 2016 at 3:35 AM, Taeung Song <treeze.taeung@gmail.com>
>>> wrote:
>>>
>>> > Hi, Joel
>>> >
>>> > perf subsystem's maintainers are
>>> > Arnaldo Carvalho de Melo <acme@kernel.org>
>>> > Peter Zijlstra <peterz@infradead.org>
>>> > Ingo Molnar <mingo@redhat.com>
>>> >
>>> > (or Namhyung Kim <namhyung@kernel.org>, Jiri Olsa <jolsa@kernel.org>)
>>> >
>>> > AFAIK,
>>> > When sending some patches, it would be better to add them into To: or Cc:
>>> >
>>> >
>>> > Thanks,
>>> > Taeung
>>> >
>>> >
>>> > On 02/28/2016 08:14 PM, Joel Fernandes wrote:
>>> >
>>> >> Sometimes for inline functions, perf probe can fail such as if an
>>> >> arguments are
>>> >> requested at probe points. This is probably because when the compiler
>>> >> inlines,
>>> >> for some instances it optimizes away arguments. Either way, the DWARF has
>>> >> missing arguments for certain probe points of inline functions causing
>>> >> 'perf
>>> >> probe' to fail. I noticed this when probing the C library that ships with
>>> >> my
>>> >> distribution. With the following patch I am successfully able to record
>>> >> kprobe
>>> >> events with arguments.
>>> >>
>>> >> Test
>>> >> ----
>>> >> ./perf probe \
>>> >> -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc
>>> >> bytes' -v
>>> >>
>>> >> Without the patch
>>> >> -----------------
>>> >> Matched function: __libc_malloc
>>> >> found inline addr: 0x831a6
>>> >> Probe point found: malloc_atfork+150
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x844a0
>>> >> Probe point found: __libc_malloc+0
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x8463b
>>> >> Probe point found: __malloc_get_state+11
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> An error occurred in debuginfo analysis (-2).
>>> >> Error: Failed to add events. Reason: No such file or directory (Code:
>>> >> -2)
>>> >>
>>> >> With the patch
>>> >> --------------
>>> >> Open Debuginfo file: /usr/lib/debug/lib/x86_64-linux-gnu/libc-2.21.so
>>> >> Try to find probe point from debuginfo.
>>> >> Symbol malloc address found : 844a0
>>> >> Matched function: __libc_malloc
>>> >> found inline addr: 0x831a6
>>> >> Probe point found: malloc_atfork+150
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x844a0
>>> >> Probe point found: __libc_malloc+0
>>> >> Searching 'bytes' variable in context.
>>> >> Converting variable bytes into trace event.
>>> >> bytes type is long unsigned int.
>>> >> found inline addr: 0x8463b
>>> >> Probe point found: __malloc_get_state+11
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> found inline addr: 0x84bba
>>> >> Probe point found: __libc_realloc+410
>>> >> Searching 'bytes' variable in context.
>>> >> Failed to find 'bytes' in this function.
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> found inline addr: 0x84d08
>>> >> ...
>>> >> Probe point error, ignoring. Atleast one probe point found
>>> >> Found 3 probe_trace_events.
>>> >> Opening /sys/kernel/debug/tracing//uprobe_events write=1
>>> >> Writing event: p:probe_libc/malloc /lib/x86_64-linux-gnu/libc-2.21.so:0x831a6
>>> >> bytes=%bp:u64
>>> >> Writing event: p:probe_libc/malloc_1 /lib/x86_64-linux-gnu/libc-2.21.so:0x844a0
>>> >> bytes=%di:u64
>>> >> Writing event: p:probe_libc/malloc_2 /lib/x86_64-linux-gnu/libc-2.21.so:0x85af7
>>> >> bytes=%di:u64
>>> >> Added new events:
>>> >> probe_libc:malloc (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >> probe_libc:malloc_1 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >> probe_libc:malloc_2 (on malloc in /lib/x86_64-linux-gnu/libc-2.21.so
>>> >> with bytes)
>>> >>
>>> >> Signed-off-by: Joel Fernandes <agnel.joel@gmail.com>
>>> >> ---
>>> >> tools/perf/util/probe-finder.c | 4 ++++
>>> >> 1 file changed, 4 insertions(+)
>>> >>
>>> >> diff --git a/tools/perf/util/probe-finder.c
>>> >> b/tools/perf/util/probe-finder.c
>>> >> index 4ce5c5e..3ac9481 100644
>>> >> --- a/tools/perf/util/probe-finder.c
>>> >> +++ b/tools/perf/util/probe-finder.c
>>> >> @@ -1255,6 +1255,10 @@ end:
>>> >> if (ret) {
>>> >> clear_probe_trace_event(tev);
>>> >> tf->ntevs--;
>>> >> + if (tf->ntevs != 0) {
>>> >> + pr_debug("Ignoring error as atleast one probe
>>> >> point found.\n");
>>> >> + ret = 0;
>>> >> + }
>>> >> }
>>> >> free(args);
>>> >> return ret;
>>> >>
>>> >>
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [RFC] perf: probe_finder: continue if atleast one probe point found
2016-03-01 4:29 ` 平松雅巳 / HIRAMATU，MASAMI
@ 2016-03-01 10:34 ` Joel Fernandes
2016-03-01 10:36 ` Joel Fernandes
0 siblings, 1 reply; 7+ messages in thread
From: Joel Fernandes @ 2016-03-01 10:34 UTC (permalink / raw
To: 平松雅巳 / HIRAMATU，MASAMI
Cc: ltc-kernel@ml.yrl.intra.hitachi.co.jp, Arnaldo Carvalho de Melo,
Taeung Song, Joel Fernandes, linux-perf-users@vger.kernel.org,
Linux Kernel Mailing List, Peter Zijlstra, Ingo Molnar,
Namhyung Kim, Jiri Olsa, sysp-manager
Hi Hiramatu,
On Mon, Feb 29, 2016 at 8:29 PM, 平松雅巳 / HIRAMATU，MASAMI
<masami.hiramatsu.pt@hitachi.com> wrote:
> BTW,
>
> ./perf probe \
> -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc $params'
>
> might help your case. $params is expanded to function parameters automatically
> and if there is no parameters, it is just ignored :)
Thanks, I agree this is a better approach.
How about an informative error letting the user know about it? Is the
below Ok? I can send a patch.
----x8--------
diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
index 4ce5c5e..21f282a 100644
--- a/tools/perf/util/probe-finder.c
+++ b/tools/perf/util/probe-finder.c
@@ -1247,8 +1247,12 @@ static int add_probe_trace_event(Dwarf_Die
*sc_die, struct probe_finder *pf)
pf->tvar = &tev->args[i];
/* Variable should be found from scope DIE */
ret = find_variable(sc_die, pf);
- if (ret != 0)
+ if (ret != 0) {
+ pr_err("Variable couldn't be found, if probe
is on inline function,"
+ "try '%s' or '%s' instead.\n",
+ PROBE_ARG_PARAMS, PROBE_ARG_VARS);
break;
+ }
}
^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [RFC] perf: probe_finder: continue if atleast one probe point found
2016-03-01 10:34 ` Joel Fernandes
@ 2016-03-01 10:36 ` Joel Fernandes
0 siblings, 0 replies; 7+ messages in thread
From: Joel Fernandes @ 2016-03-01 10:36 UTC (permalink / raw
To: 平松雅巳 / HIRAMATU，MASAMI
Cc: ltc-kernel@ml.yrl.intra.hitachi.co.jp, Arnaldo Carvalho de Melo,
Taeung Song, Joel Fernandes, linux-perf-users@vger.kernel.org,
Linux Kernel Mailing List, Peter Zijlstra, Ingo Molnar,
Namhyung Kim, Jiri Olsa, sysp-manager
Sorry about the line wrapping, I sent this email through g-mail.
Shouldn't have any issue when I send the final patch through git
send-email.
Thanks,
Joel
On Tue, Mar 1, 2016 at 2:34 AM, Joel Fernandes <agnel.joel@gmail.com> wrote:
> Hi Hiramatu,
>
> On Mon, Feb 29, 2016 at 8:29 PM, 平松雅巳 / HIRAMATU，MASAMI
> <masami.hiramatsu.pt@hitachi.com> wrote:
>> BTW,
>>
>> ./perf probe \
>> -k ./vmlinux -s ./ -x /lib/x86_64-linux-gnu/libc.so.6 -a 'malloc $params'
>>
>> might help your case. $params is expanded to function parameters automatically
>> and if there is no parameters, it is just ignored :)
>
> Thanks, I agree this is a better approach.
>
> How about an informative error letting the user know about it? Is the
> below Ok? I can send a patch.
>
> ----x8--------
> diff --git a/tools/perf/util/probe-finder.c b/tools/perf/util/probe-finder.c
> index 4ce5c5e..21f282a 100644
> --- a/tools/perf/util/probe-finder.c
> +++ b/tools/perf/util/probe-finder.c
> @@ -1247,8 +1247,12 @@ static int add_probe_trace_event(Dwarf_Die
> *sc_die, struct probe_finder *pf)
> pf->tvar = &tev->args[i];
> /* Variable should be found from scope DIE */
> ret = find_variable(sc_die, pf);
> - if (ret != 0)
> + if (ret != 0) {
> + pr_err("Variable couldn't be found, if probe
> is on inline function,"
> + "try '%s' or '%s' instead.\n",
> + PROBE_ARG_PARAMS, PROBE_ARG_VARS);
> break;
> + }
> }
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2016-03-01 10:36 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2016-02-28 11:14 [RFC] perf: probe_finder: continue if atleast one probe point found Joel Fernandes
2016-02-28 11:35 ` Taeung Song
[not found] ` <CAD=GYpZtDb3ugJRFQg1karJ_Fb7RsGPpSu0ZbL8i9Oy8x5eZ9A@mail.gmail.com>
2016-02-29 14:19 ` Arnaldo Carvalho de Melo
2016-03-01 3:09 ` 平松雅巳 / HIRAMATU，MASAMI
2016-03-01 4:29 ` 平松雅巳 / HIRAMATU，MASAMI
2016-03-01 10:34 ` Joel Fernandes
2016-03-01 10:36 ` Joel Fernandes
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.