From: Ingo Molnar <mingo@elte.hu>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: "Zhang, Yanmin" <yanmin_zhang@linux.intel.com>,
Andi Kleen <andi@firstfloor.org>, Matthew Wilcox <matthew@wil.cx>,
LKML <linux-kernel@vger.kernel.org>,
Alexander Viro <viro@ftp.linux.org.uk>,
Andrew Morton <akpm@linux-foundation.org>,
Thomas Gleixner <tglx@linutronix.de>,
"H. Peter Anvin" <hpa@zytor.com>
Subject: Re: [patch] speed up / fix the new generic semaphore code (fix AIM7 40% regression with 2.6.26-rc1)
Date: Fri, 9 May 2008 00:02:29 +0200 [thread overview]
Message-ID: <20080508220229.GA15099@elte.hu> (raw)
In-Reply-To: <20080508214557.GA13311@elte.hu>
* Ingo Molnar <mingo@elte.hu> wrote:
> yeah, i captured a trace of all the down()s that happen in the
> workload, using ftrace/sched_switch + stacktrace + a tracepoint in
> down(). Here's the semaphore activities in the trace:
i've updated the trace with a better one:
> http://redhat.com/~mingo/misc/aim7-ftrace.txt
the first one had some idle time in it as well plus the effects of a
2000-task Ctrl-Z - which skewed the histograms.
the new stats are more straightforward. down() callsite histogram:
42 down <= lock_kernel <= de_put <= proc_delete_inode <
42 down <= lock_kernel <= proc_lookup_de <= proc_lookup <
78 down <= lock_kernel <= proc_lookup_de <= proc_lookup <
310 down <= lock_kernel <= sys_fcntl <= system_call_after_swapgs <
332 down <= lock_kernel <= vfs_ioctl <= do_vfs_ioctl <
380 down <= lock_kernel <= tty_release <= __fput <
422 down <= lock_kernel <= chrdev_open <= __dentry_open <
hm, why is chrdev_open() called all that often?
top-5 wakeups:
4 default_wake_function <= __wake_up_common <= complete <= migration_thread <= kthread <= child_rip <
4 wake_up_process <= sched_exec <= do_execve <= sys_execve <= stub_execve <= <
8 wake_up_process <= __mutex_unlock_slowpath <= mutex_unlock <= do_filp_open <= do_sys_open <= sys_open <
40 default_wake_function <= autoremove_wake_function <= __wake_up_common <= __wake_up <= sock_def_wakeup <= tcp_rcv_state_process <
98 default_wake_function <= __wake_up_common <= __wake_up_sync <= do_notify_parent <= do_exit <= do_group_exit <
i.e. very little wakeup activities. Top 10 scheduling points:
10 schedule <= kjournald <= kthread <= child_rip <
12 schedule <= __down_write_nested <= __down_write <= down_write <
29 schedule <= worker_thread <= kthread <= child_rip <
40 schedule <= schedule_timeout <= inet_stream_connect <= sys_connect <
59 schedule <= __cond_resched <= _cond_resched <= generic_file_buffered_write <
111 schedule <= ksoftirqd <= kthread <= child_rip <
119 schedule <= do_wait <= sys_wait4 <= system_call_after_swapgs <
659 schedule <= do_exit <= do_group_exit <= sys_exit_group <
781 schedule <= sysret_careful <= <= 0 <
1347 schedule <= retint_careful <= <= 0 <
> and here's a few seconds worth of NMI driven readprofile output:
the NMI profiling results were accurate.
Ingo
next prev parent reply other threads:[~2008-05-08 22:03 UTC|newest]
Thread overview: 140+ messages / expand[flat|nested] mbox.gz Atom feed top
2008-05-06 5:48 AIM7 40% regression with 2.6.26-rc1 Zhang, Yanmin
2008-05-06 11:18 ` Matthew Wilcox
2008-05-06 11:44 ` Ingo Molnar
2008-05-06 12:09 ` Matthew Wilcox
2008-05-06 16:23 ` Matthew Wilcox
2008-05-06 16:36 ` Linus Torvalds
2008-05-06 16:42 ` Matthew Wilcox
2008-05-06 16:39 ` Alan Cox
2008-05-06 16:51 ` Matthew Wilcox
2008-05-06 16:45 ` Alan Cox
2008-05-06 17:42 ` Linus Torvalds
2008-05-06 20:28 ` Linus Torvalds
2008-05-06 16:44 ` J. Bruce Fields
2008-05-06 17:21 ` Andrew Morton
2008-05-06 17:31 ` Matthew Wilcox
2008-05-06 17:49 ` Ingo Molnar
2008-05-06 18:07 ` Andrew Morton
2008-05-11 11:11 ` Matthew Wilcox
2008-05-06 17:39 ` Ingo Molnar
2008-05-07 6:49 ` Zhang, Yanmin
2008-05-06 17:45 ` Linus Torvalds
2008-05-07 16:38 ` Matthew Wilcox
2008-05-07 16:55 ` Linus Torvalds
2008-05-07 17:08 ` Linus Torvalds
2008-05-07 17:16 ` Andrew Morton
2008-05-07 17:27 ` Linus Torvalds
2008-05-07 17:22 ` Ingo Molnar
2008-05-07 17:25 ` Ingo Molnar
2008-05-07 17:31 ` Linus Torvalds
2008-05-07 17:47 ` Linus Torvalds
2008-05-07 17:49 ` Ingo Molnar
2008-05-07 18:02 ` Linus Torvalds
2008-05-07 18:17 ` Ingo Molnar
2008-05-07 18:27 ` Linus Torvalds
2008-05-07 18:43 ` Ingo Molnar
2008-05-07 19:01 ` Linus Torvalds
2008-05-07 19:09 ` Ingo Molnar
2008-05-07 19:24 ` Matthew Wilcox
2008-05-07 19:44 ` Linus Torvalds
2008-05-07 20:00 ` Oi. NFS people. Read this Matthew Wilcox
2008-05-07 22:10 ` Trond Myklebust
2008-05-09 1:43 ` J. Bruce Fields
2008-05-08 3:24 ` AIM7 40% regression with 2.6.26-rc1 Zhang, Yanmin
2008-05-08 3:34 ` Linus Torvalds
2008-05-08 4:37 ` Zhang, Yanmin
2008-05-08 14:58 ` Linus Torvalds
2008-05-07 2:11 ` Zhang, Yanmin
2008-05-07 3:41 ` Zhang, Yanmin
2008-05-07 3:59 ` Andrew Morton
2008-05-07 4:46 ` Zhang, Yanmin
2008-05-07 6:26 ` Ingo Molnar
2008-05-07 6:28 ` Ingo Molnar
2008-05-07 7:05 ` Zhang, Yanmin
2008-05-07 11:00 ` Andi Kleen
2008-05-07 11:46 ` Matthew Wilcox
2008-05-07 12:21 ` Andi Kleen
2008-05-07 14:36 ` Linus Torvalds
2008-05-07 14:35 ` Alan Cox
2008-05-07 15:00 ` Linus Torvalds
2008-05-07 15:02 ` Linus Torvalds
2008-05-07 14:57 ` Andi Kleen
2008-05-07 15:31 ` Andrew Morton
2008-05-07 16:22 ` Matthew Wilcox
2008-05-07 15:19 ` Linus Torvalds
2008-05-07 17:14 ` Ingo Molnar
2008-05-08 2:44 ` Zhang, Yanmin
2008-05-08 3:29 ` Linus Torvalds
2008-05-08 4:08 ` Zhang, Yanmin
2008-05-08 4:17 ` Linus Torvalds
2008-05-08 12:01 ` [patch] speed up / fix the new generic semaphore code (fix AIM7 40% regression with 2.6.26-rc1) Ingo Molnar
2008-05-08 12:28 ` Ingo Molnar
2008-05-08 14:43 ` Ingo Molnar
2008-05-08 15:10 ` [git pull] scheduler fixes Ingo Molnar
2008-05-08 15:33 ` Adrian Bunk
2008-05-08 15:41 ` Ingo Molnar
2008-05-08 19:42 ` Adrian Bunk
2008-05-11 11:03 ` Matthew Wilcox
2008-05-11 11:14 ` Matthew Wilcox
2008-05-11 11:48 ` Matthew Wilcox
2008-05-11 12:50 ` Ingo Molnar
2008-05-11 12:52 ` Ingo Molnar
2008-05-11 13:02 ` Matthew Wilcox
2008-05-11 13:26 ` Matthew Wilcox
2008-05-11 14:00 ` Ingo Molnar
2008-05-11 14:18 ` Matthew Wilcox
2008-05-11 14:42 ` Ingo Molnar
2008-05-11 14:48 ` Matthew Wilcox
2008-05-11 15:19 ` Ingo Molnar
2008-05-11 15:29 ` Matthew Wilcox
2008-05-13 14:11 ` Ingo Molnar
2008-05-13 14:21 ` Matthew Wilcox
2008-05-13 14:42 ` Ingo Molnar
2008-05-13 15:28 ` Matthew Wilcox
2008-05-13 17:13 ` Ingo Molnar
2008-05-13 17:22 ` Linus Torvalds
2008-05-13 21:05 ` Ingo Molnar
2008-05-11 13:54 ` Ingo Molnar
2008-05-11 14:22 ` Matthew Wilcox
2008-05-11 14:32 ` Ingo Molnar
2008-05-11 14:46 ` Matthew Wilcox
2008-05-11 16:47 ` Linus Torvalds
2008-05-11 13:01 ` Ingo Molnar
2008-05-11 13:06 ` Matthew Wilcox
2008-05-11 13:45 ` Ingo Molnar
2008-05-11 14:10 ` Sven Wegener
2008-05-08 16:02 ` [patch] speed up / fix the new generic semaphore code (fix AIM7 40% regression with 2.6.26-rc1) Linus Torvalds
2008-05-08 18:30 ` Linus Torvalds
2008-05-08 20:19 ` Ingo Molnar
2008-05-08 20:27 ` Linus Torvalds
2008-05-08 21:45 ` Ingo Molnar
2008-05-08 22:02 ` Ingo Molnar [this message]
2008-05-08 22:55 ` Linus Torvalds
2008-05-08 23:07 ` Linus Torvalds
2008-05-08 23:14 ` Linus Torvalds
2008-05-08 23:16 ` Alan Cox
2008-05-08 23:33 ` Linus Torvalds
2008-05-08 23:27 ` Alan Cox
2008-05-09 6:50 ` Ingo Molnar
2008-05-09 8:29 ` Andi Kleen
2008-05-08 13:20 ` Matthew Wilcox
2008-05-08 15:01 ` Ingo Molnar
2008-05-08 13:56 ` Arjan van de Ven
2008-05-08 6:43 ` AIM7 40% regression with 2.6.26-rc1 Ingo Molnar
2008-05-08 6:48 ` Andrew Morton
2008-05-08 7:14 ` Zhang, Yanmin
2008-05-08 7:39 ` Ingo Molnar
2008-05-08 8:44 ` Zhang, Yanmin
2008-05-08 9:21 ` Ingo Molnar
2008-05-08 9:29 ` Ingo Molnar
2008-05-08 9:30 ` Zhang, Yanmin
2008-05-07 16:20 ` Ingo Molnar
2008-05-07 16:35 ` Linus Torvalds
2008-05-07 17:05 ` Ingo Molnar
2008-05-07 17:24 ` Linus Torvalds
2008-05-07 17:36 ` Ingo Molnar
2008-05-07 17:55 ` Linus Torvalds
2008-05-07 17:59 ` Matthew Wilcox
2008-05-07 18:17 ` Linus Torvalds
2008-05-07 18:49 ` Ingo Molnar
2008-05-07 13:59 ` Alan Cox
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20080508220229.GA15099@elte.hu \
--to=mingo@elte.hu \
--cc=akpm@linux-foundation.org \
--cc=andi@firstfloor.org \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=matthew@wil.cx \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=viro@ftp.linux.org.uk \
--cc=yanmin_zhang@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.