* INFO: rcu detected stall in corrupted
@ 2018-05-21 18:05 syzbot
2018-05-21 18:09 ` David Miller
0 siblings, 1 reply; 11+ messages in thread
From: syzbot @ 2018-05-21 18:05 UTC (permalink / raw
To: davem, kuznet, linux-kernel, netdev, syzkaller-bugs, yoshfuji
Hello,
syzbot found the following crash on:
HEAD commit: 771c577c23ba Linux 4.17-rc6
git tree: upstream
console output: https://syzkaller.appspot.com/x/log.txt?x=17134357800000
kernel config: https://syzkaller.appspot.com/x/.config?x=982e2df1b9e60b02
dashboard link: https://syzkaller.appspot.com/bug?extid=f116bc1994efe725d51b
compiler: gcc (GCC) 8.0.1 20180413 (experimental)
userspace arch: i386
syzkaller repro:https://syzkaller.appspot.com/x/repro.syz?x=14e5a7cf800000
IMPORTANT: if you fix the bug, please add the following tag to the commit:
Reported-by: syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com
IPv6: ADDRCONF(NETDEV_UP): veth1: link is not ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
8021q: adding VLAN 0 to HW filter on device team0
8021q: adding VLAN 0 to HW filter on device team0
INFO: rcu_sched self-detected stall on CPU
INFO: rcu_sched detected stalls on CPUs/tasks:
0-...!: (124975 ticks this GP) idle=a36/1/4611686018427387906
softirq=14002/14002 fqs=10
0-...!: (124975 ticks this GP) idle=a36/1/4611686018427387906
softirq=14002/14002 fqs=10
(t=125002 jiffies g=7347 c=7346 q=349000)
(detected by 1, t=125002 jiffies, g=7347, c=7346, q=349000)
rcu_sched kthread starved for 124927 jiffies! g7347 c7346 f0x2
RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1
Sending NMI from CPU 1 to CPUs 0:
RCU grace-period kthread stack dump:
NMI backtrace for cpu 0
CPU: 0 PID: 8 Comm: ksoftirqd/0 Not tainted 4.17.0-rc6+ #86
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
RIP: 0010:get_current arch/x86/include/asm/current.h:15 [inline]
RIP: 0010:write_comp_data+0xa/0x70 kernel/kcov.c:121
RSP: 0018:ffff8801dae06d30 EFLAGS: 00000006
RAX: 0000000000010105 RBX: 0000000000000006 RCX: ffffffff876bdc58
RDX: 0000000000000000 RSI: 0000000000000005 RDI: 0000000000000001
RBP: ffff8801dae06d68 R08: ffff8801d9a9c200 R09: fffffbfff14da4bc
R10: fffffbfff14da4bc R11: ffffffff8a6d25e0 R12: ffffffff88644220
R13: 0000000000000000 R14: 0000000000000001 R15: 0000000000000008
FS: 0000000000000000(0000) GS:ffff8801dae00000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f595e194270 CR3: 00000001b09c2000 CR4: 00000000001406f0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
Call Trace:
<IRQ>
vsnprintf+0x1b8/0x1b40 lib/vsprintf.c:2252
sprintf+0xa7/0xd0 lib/vsprintf.c:2498
print_time kernel/printk/printk.c:1223 [inline]
print_prefix+0x26a/0x3f0 kernel/printk/printk.c:1246
msg_print_text+0xca/0x1c0 kernel/printk/printk.c:1273
console_unlock+0x4f5/0x1100 kernel/printk/printk.c:2369
vprintk_emit+0x6ad/0xdd0 kernel/printk/printk.c:1907
vprintk_default+0x28/0x30 kernel/printk/printk.c:1947
vprintk_func+0x7a/0xe7 kernel/printk/printk_safe.c:379
printk+0x9e/0xba kernel/printk/printk.c:1980
rcu_check_gp_kthread_starvation+0x325/0x3a4 kernel/rcu/tree.c:1353
print_cpu_stall kernel/rcu/tree.c:1523 [inline]
check_cpu_stall.isra.61.cold.80+0x364/0x59a kernel/rcu/tree.c:1593
__rcu_pending kernel/rcu/tree.c:3356 [inline]
rcu_pending kernel/rcu/tree.c:3401 [inline]
rcu_check_callbacks+0x21b/0xad0 kernel/rcu/tree.c:2763
update_process_times+0x2d/0x70 kernel/time/timer.c:1636
tick_sched_handle+0x9f/0x180 kernel/time/tick-sched.c:164
tick_sched_timer+0x45/0x130 kernel/time/tick-sched.c:1274
__run_hrtimer kernel/time/hrtimer.c:1398 [inline]
__hrtimer_run_queues+0x3e3/0x10a0 kernel/time/hrtimer.c:1460
hrtimer_interrupt+0x2f3/0x750 kernel/time/hrtimer.c:1518
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1025 [inline]
smp_apic_timer_interrupt+0x15d/0x710 arch/x86/kernel/apic/apic.c:1050
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:863
</IRQ>
RIP: 0010:__sanitizer_cov_trace_pc+0x1/0x50 kernel/kcov.c:94
RSP: 0018:ffff8801d9aad680 EFLAGS: 00000293 ORIG_RAX: ffffffffffffff13
RAX: 0000000000000103 RBX: 0000000000000002 RCX: ffffffff867e02e0
RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000005
RBP: ffff8801d9aad7e0 R08: ffff8801d9a9c200 R09: ffff8801d9aadaf0
R10: ffffed003b5c46c2 R11: ffff8801dae23613 R12: ffff8801ce597c40
R13: 0000000000000000 R14: 0000000000000002 R15: 0000000000000000
find_match+0x244/0x13a0 net/ipv6/route.c:691
find_rr_leaf net/ipv6/route.c:729 [inline]
rt6_select net/ipv6/route.c:779 [inline]
ip6_pol_route+0x946/0x3d40 net/ipv6/route.c:1705
ip6_pol_route_output+0x54/0x70 net/ipv6/route.c:1969
fib6_rule_lookup+0x211/0x6d0 net/ipv6/fib6_rules.c:89
ip6_route_output_flags+0x2c5/0x350 net/ipv6/route.c:1997
ip6_dst_lookup_tail+0x47b/0x1b30 net/ipv6/ip6_output.c:995
ip6_dst_lookup_flow+0xc1/0x260 net/ipv6/ip6_output.c:1096
sctp_v6_get_dst+0x16b4/0x20b0 net/sctp/ipv6.c:327
sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
? trace_hardirqs_off
Lost 148 message(s)!
rcu_sched kthread starved for 124927 jiffies! g7347 c7346 f0x2
RCU_GP_WAIT_FQS(3) ->state=0x0 ->cpu=1
rcu_sched R
RCU grace-period kthread stack dump:
rcu_sched R
running task
running task 23896 9 2 0x80000000
23896 9 2 0x80000000
Call Trace:
Call Trace:
context_switch kernel/sched/core.c:2859 [inline]
__schedule+0x801/0x1e30 kernel/sched/core.c:3501
context_switch kernel/sched/core.c:2859 [inline]
__schedule+0x801/0x1e30 kernel/sched/core.c:3501
schedule+0xef/0x430 kernel/sched/core.c:3545
schedule+0xef/0x430 kernel/sched/core.c:3545
schedule_timeout+0x138/0x240 kernel/time/timer.c:1801
schedule_timeout+0x138/0x240 kernel/time/timer.c:1801
rcu_gp_kthread+0x6b5/0x1940 kernel/rcu/tree.c:2231
rcu_gp_kthread+0x6b5/0x1940 kernel/rcu/tree.c:2231
kthread+0x345/0x410 kernel/kthread.c:240
kthread+0x345/0x410 kernel/kthread.c:240
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412
NMI backtrace for cpu 0
CPU: 0 PID: 8 Comm: ksoftirqd/0 Not tainted 4.17.0-rc6+ #86
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
Google 01/01/2011
Call Trace:
<IRQ>
__dump_stack lib/dump_stack.c:77 [inline]
dump_stack+0x1b9/0x294 lib/dump_stack.c:113
nmi_cpu_backtrace.cold.4+0x19/0xce lib/nmi_backtrace.c:103
nmi_trigger_cpumask_backtrace+0x151/0x192 lib/nmi_backtrace.c:62
arch_trigger_cpumask_backtrace+0x14/0x20 arch/x86/kernel/apic/hw_nmi.c:38
trigger_single_cpu_backtrace include/linux/nmi.h:156 [inline]
rcu_dump_cpu_stacks+0x175/0x1c2 kernel/rcu/tree.c:1376
print_cpu_stall kernel/rcu/tree.c:1525 [inline]
check_cpu_stall.isra.61.cold.80+0x36c/0x59a kernel/rcu/tree.c:1593
__rcu_pending kernel/rcu/tree.c:3356 [inline]
rcu_pending kernel/rcu/tree.c:3401 [inline]
rcu_check_callbacks+0x21b/0xad0 kernel/rcu/tree.c:2763
update_process_times+0x2d/0x70 kernel/time/timer.c:1636
tick_sched_handle+0x9f/0x180 kernel/time/tick-sched.c:164
tick_sched_timer+0x45/0x130 kernel/time/tick-sched.c:1274
__run_hrtimer kernel/time/hrtimer.c:1398 [inline]
__hrtimer_run_queues+0x3e3/0x10a0 kernel/time/hrtimer.c:1460
hrtimer_interrupt+0x2f3/0x750 kernel/time/hrtimer.c:1518
local_apic_timer_interrupt arch/x86/kernel/apic/apic.c:1025 [inline]
smp_apic_timer_interrupt+0x15d/0x710 arch/x86/kernel/apic/apic.c:1050
apic_timer_interrupt+0xf/0x20 arch/x86/entry/entry_64.S:863
</IRQ>
RIP: 0010:__sanitizer_cov_trace_pc+0x1/0x50 kernel/kcov.c:94
RSP: 0018:ffff8801d9aad680 EFLAGS: 00000293 ORIG_RAX: ffffffffffffff13
RAX: 0000000000000103 RBX: 0000000000000002 RCX: ffffffff867e02e0
RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000005
RBP: ffff8801d9aad7e0 R08: ffff8801d9a9c200 R09: ffff8801d9aadaf0
R10: ffffed003b5c46c2 R11: ffff8801dae23613 R12: ffff8801ce597c40
R13: 0000000000000000 R14: 0000000000000002 R15: 0000000000000000
find_match+0x244/0x13a0 net/ipv6/route.c:691
find_rr_leaf net/ipv6/route.c:729 [inline]
rt6_select net/ipv6/route.c:779 [inline]
ip6_pol_route+0x946/0x3d40 net/ipv6/route.c:1705
ip6_pol_route_output+0x54/0x70 net/ipv6/route.c:1969
fib6_rule_lookup+0x211/0x6d0 net/ipv6/fib6_rules.c:89
ip6_route_output_flags+0x2c5/0x350 net/ipv6/route.c:1997
ip6_dst_lookup_tail+0x47b/0x1b30 net/ipv6/ip6_output.c:995
ip6_dst_lookup_flow+0xc1/0x260 net/ipv6/ip6_output.c:1096
sctp_v6_get_dst+0x16b4/0x20b0 net/sctp/ipv6.c:327
sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
expire_timers kernel/time/timer.c:1363 [inline]
__run_timers+0x79e/0xc50 kernel/time/timer.c:1666
run_timer_softirq+0x4c/0x70 kernel/time/timer.c:1692
__do_softirq+0x2e0/0xaf5 kernel/softirq.c:285
run_ksoftirqd+0x86/0x100 kernel/softirq.c:646
smpboot_thread_fn+0x417/0x870 kernel/smpboot.c:164
kthread+0x345/0x410 kernel/kthread.c:240
ret_from_fork+0x3a/0x50 arch/x86/entry/entry_64.S:412
INFO: NMI handler (nmi_cpu_backtrace_handler) took too long to run: 1.929
msecs
BUG: workqueue lockup - pool cpus=0-1 flags=0x4 nice=0 stuck for 127s!
Showing busy workqueues and worker pools:
workqueue events: flags=0x0
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=10/256
pending: pcpu_balance_workfn, defense_work_handler,
defense_work_handler, defense_work_handler, defense_work_handler,
defense_work_handler, defense_work_handler, check_corruption,
vmstat_shepherd, cache_reap
workqueue events_power_efficient: flags=0x80
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256
pending: check_lifetime
workqueue mm_percpu_wq: flags=0x8
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/256
pending: vmstat_update
workqueue writeback: flags=0x4e
pwq 4: cpus=0-1 flags=0x4 nice=0 active=4/256
pending: wb_workfn, wb_workfn, wb_workfn, wb_workfn
workqueue kblockd: flags=0x18
pwq 1: cpus=0 node=0 flags=0x0 nice=-20 active=1/256
pending: blk_mq_timeout_work
workqueue ib_addr: flags=0xa0002
pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/1
pending: process_req
workqueue gid-cache-wq: flags=0xa0002
pwq 4: cpus=0-1 flags=0x4 nice=0 active=1/1
pending: update_gid_event_work_handler
delayed: netdevice_event_work_handler
workqueue ipv6_addrconf: flags=0x40008
pwq 0: cpus=0 node=0 flags=0x0 nice=0 active=1/1
pending: addrconf_dad_work
delayed: addrconf_dad_work, addrconf_dad_work, addrconf_dad_work,
addrconf_dad_work, addrconf_dad_work, addrconf_dad_work, addrconf_dad_work,
addrconf_dad_work, addrconf_dad_work, addrconf_dad_work, addrconf_dad_work,
addrconf_dad_work, addrconf_dad_work, addrconf_dad_work, addrconf_dad_work,
addrconf_dad_work, addrconf_dad_work, addrconf_dad_work
---
This bug is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this bug report. See:
https://goo.gl/tpsmEJ#bug-status-tracking for how to communicate with
syzbot.
syzbot can test patches for this bug, for details see:
https://goo.gl/tpsmEJ#testing-patches
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
2018-05-21 18:05 syzbot
@ 2018-05-21 18:09 ` David Miller
2018-05-21 18:13 ` Eric Dumazet
0 siblings, 1 reply; 11+ messages in thread
From: David Miller @ 2018-05-21 18:09 UTC (permalink / raw
To: syzbot+f116bc1994efe725d51b
Cc: kuznet, linux-kernel, netdev, syzkaller-bugs, yoshfuji, dsahern,
roopa
From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
Date: Mon, 21 May 2018 11:05:02 -0700
> find_match+0x244/0x13a0 net/ipv6/route.c:691
> find_rr_leaf net/ipv6/route.c:729 [inline]
> rt6_select net/ipv6/route.c:779 [inline]
Hmmm, endless loop in find_rr_leaf or similar?
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
2018-05-21 18:09 ` David Miller
@ 2018-05-21 18:13 ` Eric Dumazet
2018-05-21 18:15 ` David Ahern
2018-05-23 23:13 ` Marcelo Ricardo Leitner
0 siblings, 2 replies; 11+ messages in thread
From: Eric Dumazet @ 2018-05-21 18:13 UTC (permalink / raw
To: David Miller, syzbot+f116bc1994efe725d51b
Cc: kuznet, linux-kernel, netdev, syzkaller-bugs, yoshfuji, dsahern,
roopa
On 05/21/2018 11:09 AM, David Miller wrote:
> From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
> Date: Mon, 21 May 2018 11:05:02 -0700
>
>> find_match+0x244/0x13a0 net/ipv6/route.c:691
>> find_rr_leaf net/ipv6/route.c:729 [inline]
>> rt6_select net/ipv6/route.c:779 [inline]
>
> Hmmm, endless loop in find_rr_leaf or similar?
>
I do not think so, this really looks like SCTP specific
, we now have dozens of traces all sharing :
sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
Some kind of infinite loop.
When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
2018-05-21 18:13 ` Eric Dumazet
@ 2018-05-21 18:15 ` David Ahern
2018-05-23 23:13 ` Marcelo Ricardo Leitner
1 sibling, 0 replies; 11+ messages in thread
From: David Ahern @ 2018-05-21 18:15 UTC (permalink / raw
To: Eric Dumazet, David Miller, syzbot+f116bc1994efe725d51b
Cc: kuznet, linux-kernel, netdev, syzkaller-bugs, yoshfuji, dsahern,
roopa
On 5/21/18 12:13 PM, Eric Dumazet wrote:
>
>
> On 05/21/2018 11:09 AM, David Miller wrote:
>> From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
>> Date: Mon, 21 May 2018 11:05:02 -0700
>>
>>> find_match+0x244/0x13a0 net/ipv6/route.c:691
>>> find_rr_leaf net/ipv6/route.c:729 [inline]
>>> rt6_select net/ipv6/route.c:779 [inline]
>>
>> Hmmm, endless loop in find_rr_leaf or similar?
>>
>
>
> I do not think so, this really looks like SCTP specific
> , we now have dozens of traces all sharing :
>
> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>
>
> Some kind of infinite loop.
>
> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
>
Just so we are looking at the right tree, the original message shows:
> HEAD commit: 771c577c23ba Linux 4.17-rc6
> git tree: upstream
I take that to mean this is Linus' tree.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
2018-05-21 18:13 ` Eric Dumazet
@ 2018-05-23 23:13 ` Marcelo Ricardo Leitner
2018-05-23 23:13 ` Marcelo Ricardo Leitner
1 sibling, 0 replies; 11+ messages in thread
From: Marcelo Ricardo Leitner @ 2018-05-23 23:13 UTC (permalink / raw
To: Eric Dumazet
Cc: David Miller, syzbot+f116bc1994efe725d51b, kuznet, linux-kernel,
netdev, syzkaller-bugs, yoshfuji, dsahern, roopa, linux-sctp,
Xin Long
On Mon, May 21, 2018 at 11:13:46AM -0700, Eric Dumazet wrote:
>
>
> On 05/21/2018 11:09 AM, David Miller wrote:
> > From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
> > Date: Mon, 21 May 2018 11:05:02 -0700
> >
> >> find_match+0x244/0x13a0 net/ipv6/route.c:691
> >> find_rr_leaf net/ipv6/route.c:729 [inline]
> >> rt6_select net/ipv6/route.c:779 [inline]
> >
> > Hmmm, endless loop in find_rr_leaf or similar?
> >
>
>
> I do not think so, this really looks like SCTP specific
> , we now have dozens of traces all sharing :
>
> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>
>
> Some kind of infinite loop.
>
> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
Agreed. Xin Long identified the root cause. syzkaller is setting too
aggressive parameters to SCTP RTO, leading to issues with the
heartbeat timer.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
@ 2018-05-23 23:13 ` Marcelo Ricardo Leitner
0 siblings, 0 replies; 11+ messages in thread
From: Marcelo Ricardo Leitner @ 2018-05-23 23:13 UTC (permalink / raw
To: Eric Dumazet
Cc: David Miller, syzbot+f116bc1994efe725d51b, kuznet, linux-kernel,
netdev, syzkaller-bugs, yoshfuji, dsahern, roopa, linux-sctp,
Xin Long
On Mon, May 21, 2018 at 11:13:46AM -0700, Eric Dumazet wrote:
>
>
> On 05/21/2018 11:09 AM, David Miller wrote:
> > From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
> > Date: Mon, 21 May 2018 11:05:02 -0700
> >
> >> find_match+0x244/0x13a0 net/ipv6/route.c:691
> >> find_rr_leaf net/ipv6/route.c:729 [inline]
> >> rt6_select net/ipv6/route.c:779 [inline]
> >
> > Hmmm, endless loop in find_rr_leaf or similar?
> >
>
>
> I do not think so, this really looks like SCTP specific
> , we now have dozens of traces all sharing :
>
> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>
>
> Some kind of infinite loop.
>
> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
Agreed. Xin Long identified the root cause. syzkaller is setting too
aggressive parameters to SCTP RTO, leading to issues with the
heartbeat timer.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
2018-05-23 23:13 ` Marcelo Ricardo Leitner
@ 2018-05-24 9:02 ` Xin Long
-1 siblings, 0 replies; 11+ messages in thread
From: Xin Long @ 2018-05-24 9:02 UTC (permalink / raw
To: Marcelo Ricardo Leitner
Cc: Eric Dumazet, David Miller, syzbot+f116bc1994efe725d51b, kuznet,
LKML, network dev, syzkaller-bugs, yoshfuji, dsahern,
Roopa Prabhu, linux-sctp
On Thu, May 24, 2018 at 7:13 AM, Marcelo Ricardo Leitner
<marcelo.leitner@gmail.com> wrote:
> On Mon, May 21, 2018 at 11:13:46AM -0700, Eric Dumazet wrote:
>>
>>
>> On 05/21/2018 11:09 AM, David Miller wrote:
>> > From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
>> > Date: Mon, 21 May 2018 11:05:02 -0700
>> >
>> >> find_match+0x244/0x13a0 net/ipv6/route.c:691
>> >> find_rr_leaf net/ipv6/route.c:729 [inline]
>> >> rt6_select net/ipv6/route.c:779 [inline]
>> >
>> > Hmmm, endless loop in find_rr_leaf or similar?
>> >
>>
>>
>> I do not think so, this really looks like SCTP specific
>> , we now have dozens of traces all sharing :
>>
>> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
>> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
>> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
>> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
>> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
>> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
>> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
>> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
>> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>>
>>
>> Some kind of infinite loop.
>>
>> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
>
> Agreed. Xin Long identified the root cause. syzkaller is setting too
> aggressive parameters to SCTP RTO, leading to issues with the
> heartbeat timer.
Right, I will prepare a fix soon with your suggestion rto_min value "HZ/5"
Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
@ 2018-05-24 9:02 ` Xin Long
0 siblings, 0 replies; 11+ messages in thread
From: Xin Long @ 2018-05-24 9:02 UTC (permalink / raw
To: Marcelo Ricardo Leitner
Cc: Eric Dumazet, David Miller, syzbot+f116bc1994efe725d51b, kuznet,
LKML, network dev, syzkaller-bugs, yoshfuji, dsahern,
Roopa Prabhu, linux-sctp
On Thu, May 24, 2018 at 7:13 AM, Marcelo Ricardo Leitner
<marcelo.leitner@gmail.com> wrote:
> On Mon, May 21, 2018 at 11:13:46AM -0700, Eric Dumazet wrote:
>>
>>
>> On 05/21/2018 11:09 AM, David Miller wrote:
>> > From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
>> > Date: Mon, 21 May 2018 11:05:02 -0700
>> >
>> >> find_match+0x244/0x13a0 net/ipv6/route.c:691
>> >> find_rr_leaf net/ipv6/route.c:729 [inline]
>> >> rt6_select net/ipv6/route.c:779 [inline]
>> >
>> > Hmmm, endless loop in find_rr_leaf or similar?
>> >
>>
>>
>> I do not think so, this really looks like SCTP specific
>> , we now have dozens of traces all sharing :
>>
>> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
>> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
>> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
>> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
>> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
>> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
>> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
>> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
>> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>>
>>
>> Some kind of infinite loop.
>>
>> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
>
> Agreed. Xin Long identified the root cause. syzkaller is setting too
> aggressive parameters to SCTP RTO, leading to issues with the
> heartbeat timer.
Right, I will prepare a fix soon with your suggestion rto_min value "HZ/5"
Thanks.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
2018-05-24 9:02 ` Xin Long
@ 2018-05-26 15:28 ` Dmitry Vyukov
-1 siblings, 0 replies; 11+ messages in thread
From: Dmitry Vyukov @ 2018-05-26 15:28 UTC (permalink / raw
To: Xin Long
Cc: Marcelo Ricardo Leitner, Eric Dumazet, David Miller,
syzbot+f116bc1994efe725d51b, kuznet, LKML, network dev,
syzkaller-bugs, yoshfuji, dsahern, Roopa Prabhu, linux-sctp
On Thu, May 24, 2018 at 11:02 AM, Xin Long <lucien.xin@gmail.com> wrote:
> On Thu, May 24, 2018 at 7:13 AM, Marcelo Ricardo Leitner
> <marcelo.leitner@gmail.com> wrote:
>> On Mon, May 21, 2018 at 11:13:46AM -0700, Eric Dumazet wrote:
>>>
>>>
>>> On 05/21/2018 11:09 AM, David Miller wrote:
>>> > From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
>>> > Date: Mon, 21 May 2018 11:05:02 -0700
>>> >
>>> >> find_match+0x244/0x13a0 net/ipv6/route.c:691
>>> >> find_rr_leaf net/ipv6/route.c:729 [inline]
>>> >> rt6_select net/ipv6/route.c:779 [inline]
>>> >
>>> > Hmmm, endless loop in find_rr_leaf or similar?
>>> >
>>>
>>>
>>> I do not think so, this really looks like SCTP specific
>>> , we now have dozens of traces all sharing :
>>>
>>> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
>>> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
>>> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
>>> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
>>> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
>>> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
>>> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
>>> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
>>> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>>>
>>>
>>> Some kind of infinite loop.
>>>
>>> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
>>
>> Agreed. Xin Long identified the root cause. syzkaller is setting too
>> aggressive parameters to SCTP RTO, leading to issues with the
>> heartbeat timer.
> Right, I will prepare a fix soon with your suggestion rto_min value "HZ/5"
> Thanks.
#syz fix: sctp: not allow to set rto_min with a value below 200 msecs
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: INFO: rcu detected stall in corrupted
@ 2018-05-26 15:28 ` Dmitry Vyukov
0 siblings, 0 replies; 11+ messages in thread
From: Dmitry Vyukov @ 2018-05-26 15:28 UTC (permalink / raw
To: Xin Long
Cc: Marcelo Ricardo Leitner, Eric Dumazet, David Miller,
syzbot+f116bc1994efe725d51b, kuznet, LKML, network dev,
syzkaller-bugs, yoshfuji, dsahern, Roopa Prabhu, linux-sctp
On Thu, May 24, 2018 at 11:02 AM, Xin Long <lucien.xin@gmail.com> wrote:
> On Thu, May 24, 2018 at 7:13 AM, Marcelo Ricardo Leitner
> <marcelo.leitner@gmail.com> wrote:
>> On Mon, May 21, 2018 at 11:13:46AM -0700, Eric Dumazet wrote:
>>>
>>>
>>> On 05/21/2018 11:09 AM, David Miller wrote:
>>> > From: syzbot <syzbot+f116bc1994efe725d51b@syzkaller.appspotmail.com>
>>> > Date: Mon, 21 May 2018 11:05:02 -0700
>>> >
>>> >> find_match+0x244/0x13a0 net/ipv6/route.c:691
>>> >> find_rr_leaf net/ipv6/route.c:729 [inline]
>>> >> rt6_select net/ipv6/route.c:779 [inline]
>>> >
>>> > Hmmm, endless loop in find_rr_leaf or similar?
>>> >
>>>
>>>
>>> I do not think so, this really looks like SCTP specific
>>> , we now have dozens of traces all sharing :
>>>
>>> sctp_transport_route+0xad/0x450 net/sctp/transport.c:293
>>> sctp_packet_config+0xb89/0xfd0 net/sctp/output.c:123
>>> sctp_outq_flush+0x79c/0x4370 net/sctp/outqueue.c:894
>>> sctp_outq_uncork+0x6a/0x80 net/sctp/outqueue.c:776
>>> sctp_cmd_interpreter net/sctp/sm_sideeffect.c:1820 [inline]
>>> sctp_side_effects net/sctp/sm_sideeffect.c:1220 [inline]
>>> sctp_do_sm+0x596/0x7160 net/sctp/sm_sideeffect.c:1191
>>> sctp_generate_heartbeat_event+0x218/0x450 net/sctp/sm_sideeffect.c:406
>>> call_timer_fn+0x230/0x940 kernel/time/timer.c:1326
>>>
>>>
>>> Some kind of infinite loop.
>>>
>>> When the hrtimer fires, it can point to any code that sits below but does not necessarily have a bug.
>>
>> Agreed. Xin Long identified the root cause. syzkaller is setting too
>> aggressive parameters to SCTP RTO, leading to issues with the
>> heartbeat timer.
> Right, I will prepare a fix soon with your suggestion rto_min value "HZ/5"
> Thanks.
#syz fix: sctp: not allow to set rto_min with a value below 200 msecs
^ permalink raw reply [flat|nested] 11+ messages in thread
* INFO: rcu detected stall in corrupted
@ 2024-03-16 3:25 cheung wall
0 siblings, 0 replies; 11+ messages in thread
From: cheung wall @ 2024-03-16 3:25 UTC (permalink / raw
To: Bjorn Helgaas; +Cc: linux-pci, linux-kernel
Hello,
when using Healer to fuzz the latest Linux Kernel, the following crash
was triggered on:
HEAD commit: 0dd3ee31125508cd67f7e7172247f05b7fd1753a (tag: v6.7)
git tree: upstream
console output: https://pastebin.com/raw/1JtKSypv
kernel config: https://pastebin.com/raw/VecrLrRN
C reproducer: https://pastebin.com/raw/CUsatTHW
Syzlang reproducer: https://pastebin.com/raw/gJFKLvkN
If you fix this issue, please add the following tag to the commit:
Reported-by: Qiang Zhang <zzqq0103.hey@gmail.com>
----------------------------------------------------------
rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
rcu: 1-...0: (0 ticks this GP) idle=84e4/1/0x4000000000000000
softirq=698253/698254 fqs=4779
rcu: (detected by 3, t=21002 jiffies, g=1119041, q=13818 ncpus=8)
Sending NMI from CPU 3 to CPUs 1:
NMI backtrace for cpu 1
CPU: 1 PID: 510326 Comm: syz-executor.1 Not tainted 6.7.0 #2
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.15.0-1 04/01/2014
RIP: 0010:vga_put+0x42/0x100 drivers/pci/vgaarb.c:547
Code: 85 e4 0f 84 aa 00 00 00 e8 8b 59 38 ff 48 c7 c7 00 31 71 b9 48
bb 00 00 00 00 00 fc ff df e8 75 c0 d0 01 48 8b 2d fe a1 f9 02 <49> 89
c6 48 81 fd 00 ba 1f b8 75 25 eb 59 e8 5b 59 38 ff 48 89 e8
RSP: 0018:ffff888117917e28 EFLAGS: 00000082
RAX: 0000000000000093 RBX: dffffc0000000000 RCX: ffffffffb6f6d916
RDX: ffff8881138ea200 RSI: 0000000000000004 RDI: ffff888117917dc8
RBP: ffff888100fd5000 R08: 0000000000000001 R09: ffffed1022f22fb9
R10: 0000000000000003 R11: 0000000000000000 R12: ffff888100ee8000
R13: 0000000000000001 R14: ffff888116747818 R15: dffffc0000000000
FS: 0000555556ccd480(0000) GS:ffff8881f7080000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f7b9128aa4f CR3: 0000000114c00003 CR4: 0000000000770ef0
PKRU: 55555554
Call Trace:
<NMI>
</NMI>
<TASK>
vga_arb_release+0x19a/0x2e0 drivers/pci/vgaarb.c:1455
__fput+0x235/0xb20 fs/file_table.c:394
__fput_sync+0x35/0x40 fs/file_table.c:475
__do_sys_close fs/open.c:1587 [inline]
__se_sys_close fs/open.c:1572 [inline]
__x64_sys_close+0x87/0xf0 fs/open.c:1572
do_syscall_x64 arch/x86/entry/common.c:52 [inline]
do_syscall_64+0x46/0xf0 arch/x86/entry/common.c:83
entry_SYSCALL_64_after_hwframe+0x6f/0x77
RIP: 0033:0x7f7b91217f0b
Code: 03 00 00 00 0f 05 48 3d 00 f0 ff ff 77 41 c3 48 83 ec 18 89 7c
24 0c e8 f3 fb 02 00 8b 7c 24 0c 41 89 c0 b8 03 00 00 00 0f 05 <48> 3d
00 f0 ff ff 77 35 44 89 c7 89 44 24 0c e8 41 fc 02 00 8b 44
RSP: 002b:00007fffbef11d10 EFLAGS: 00000293 ORIG_RAX: 0000000000000003
RAX: ffffffffffffffda RBX: 0000000000000004 RCX: 00007f7b91217f0b
RDX: 0000000000000000 RSI: ffffffffffffffff RDI: 0000000000000003
RBP: 0000000000000000 R08: 0000000000000000 R09: 00007f7b90d89bd8
R10: 00007fffbef11e10 R11: 0000000000000293 R12: 00007f7b90d89000
R13: 00007fffbef11e10 R14: 00007f7b90d89c80 R15: 00007f7b90d89c78
</TASK>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-03-16 3:26 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-03-16 3:25 INFO: rcu detected stall in corrupted cheung wall
-- strict thread matches above, loose matches on Subject: below --
2018-05-21 18:05 syzbot
2018-05-21 18:09 ` David Miller
2018-05-21 18:13 ` Eric Dumazet
2018-05-21 18:15 ` David Ahern
2018-05-23 23:13 ` Marcelo Ricardo Leitner
2018-05-23 23:13 ` Marcelo Ricardo Leitner
2018-05-24 9:02 ` Xin Long
2018-05-24 9:02 ` Xin Long
2018-05-26 15:28 ` Dmitry Vyukov
2018-05-26 15:28 ` Dmitry Vyukov
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.