From: Jim Paris <jim@jtan.com>
To: linux-hotplug@vger.kernel.org, linux-kernel@vger.kernel.org
Cc: "Eric W. Biederman" <ebiederm@xmission.com>,
Milan Broz <mbroz@redhat.com>
Subject: udev missing events?
Date: Sun, 22 Apr 2012 04:36:55 +0000 [thread overview]
Message-ID: <20120422043655.GA27861@psychosis.jim.sh> (raw)
(cc'd Eric and Milan because 5f71a296 and ebf4127c might be related)
I'm trying to track down a problem that started with virt-manager not
being able to connect to libvirtd. Long story short, the problem is
that "udevadm settle" is timing out, with no events in the udev queue.
(I know it's bad for libvirt to rely on "udevadm settle", but it seems
that this is exposing some other problem).
This is with 3.2.14 from Debian.
I can trigger the problem easily by just starting libvirtd and killing
it. Then, the counters are stuck here:
# cat /sys/kernel/uevent_seqnum
13593
# od -t d2 /dev/.udev/queue.bin | head -1
0000000 13592
During the libvirtd startup, I ran "udevadm monitor", which seems to
indicate that 13593 never made it to udev:
# udevadm monitor --kernel --property
monitor will print the received events for:
KERNEL - the kernel uevent
KERNEL[537903.765581] add /devices/virtual/net/lo/queues/rx-0 (queues)
ACTIONd
DEVPATH=/devices/virtual/net/lo/queues/rx-0
SEQNUM\x13589
SUBSYSTEM=queues
UDEV_LOG=3
KERNEL[537903.765683] add /devices/virtual/net/lo/queues/tx-0 (queues)
ACTIONd
DEVPATH=/devices/virtual/net/lo/queues/tx-0
SEQNUM\x13590
SUBSYSTEM=queues
UDEV_LOG=3
KERNEL[537903.844154] remove /devices/virtual/net/lo/queues/rx-0 (queues)
ACTION=remove
DEVPATH=/devices/virtual/net/lo/queues/rx-0
SEQNUM\x13591
SUBSYSTEM=queues
UDEV_LOG=3
KERNEL[537903.844192] remove /devices/virtual/net/lo/queues/tx-0 (queues)
ACTION=remove
DEVPATH=/devices/virtual/net/lo/queues/tx-0
SEQNUM\x13592
SUBSYSTEM=queues
UDEV_LOG=3
To try to track it further, I wrote a systemtap script (available at
http://jim.sh/~jim/tmp/monitor.stp). It dumps the result of
netlink_broadcast_filtered at lib/kobject_uevent.c:248, and then dumps
the uevent_seqnum and environment at the end of the same function.
It seems to show that netlink_broadcast_filter is failing to send out
the event, with -ESRCH:
$ sudo ./monitor.stp
begin
-----
netlink_broadcast_filtered: -3 (ESRCH)
netlink_broadcast_filtered: -3 (ESRCH)
event was:
uevent_seqnum: 13588
"ACTIONd"
"DEVPATH=/devices/virtual/net/lo"
"SUBSYSTEM=net"
"INTERFACE=lo"
"IFINDEXh"
"SEQNUM\x13588"
-----
netlink_broadcast_filtered: 0x0
netlink_broadcast_filtered: -3 (ESRCH)
event was:
uevent_seqnum: 13589
"ACTIONd"
"DEVPATH=/devices/virtual/net/lo/queues/rx-0"
"SUBSYSTEM=queues"
"SEQNUM\x13589"
-----
netlink_broadcast_filtered: 0x0
netlink_broadcast_filtered: -3 (ESRCH)
event was:
uevent_seqnum: 13590
"ACTIONd"
"DEVPATH=/devices/virtual/net/lo/queues/tx-0"
"SUBSYSTEM=queues"
"SEQNUM\x13590"
-----
netlink_broadcast_filtered: 0x0
netlink_broadcast_filtered: -3 (ESRCH)
event was:
uevent_seqnum: 13591
"ACTION=remove"
"DEVPATH=/devices/virtual/net/lo/queues/rx-0"
"SUBSYSTEM=queues"
"SEQNUM\x13591"
-----
netlink_broadcast_filtered: 0x0
netlink_broadcast_filtered: -3 (ESRCH)
event was:
uevent_seqnum: 13592
"ACTION=remove"
"DEVPATH=/devices/virtual/net/lo/queues/tx-0"
"SUBSYSTEM=queues"
"SEQNUM\x13592"
-----
netlink_broadcast_filtered: -3 (ESRCH)
netlink_broadcast_filtered: -3 (ESRCH)
event was:
uevent_seqnum: 13593
"ACTION=remove"
"DEVPATH=/devices/virtual/net/lo"
"SUBSYSTEM=net"
"INTERFACE=lo"
"IFINDEXh"
"SEQNUM\x13593"
-----
^Cend
Is it expected that those events would not be sent like that?
If not, is there something else I can do to track this further?
Or, if that's expected behavior, is there anything "udevadm settle"
can do to avoid the timeout?
-jim
next reply other threads:[~2012-04-22 4:36 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-04-22 4:36 Jim Paris [this message]
[not found] ` <ef67a489-94bc-4f97-903d-907ab974537c@email.android.com>
2012-04-22 14:11 ` udev missing events? Jim Paris
2012-04-22 16:07 ` Eric W. Biederman
2012-04-22 16:54 ` Jim Paris
2012-04-27 13:59 ` Frank Ch. Eigler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120422043655.GA27861@psychosis.jim.sh \
--to=jim@jtan.com \
--cc=ebiederm@xmission.com \
--cc=linux-hotplug@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mbroz@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).