* [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
@ 2024-03-19 18:12 syzbot
2024-03-21 7:33 ` stack access issue. " Alexei Starovoitov
0 siblings, 1 reply; 8+ messages in thread
From: syzbot @ 2024-03-19 18:12 UTC (permalink / raw
To: andrii, ast, bpf, daniel, eddyz87, haoluo, john.fastabend, jolsa,
kpsingh, linux-kernel, martin.lau, netdev, sdf, song,
syzkaller-bugs, yonghong.song
Hello,
syzbot found the following issue on:
HEAD commit: 0740b6427e90 Merge branch 'bpf-arena-followups'
git tree: bpf
console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
------------[ cut here ]------------
UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
Call Trace:
<TASK>
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
ubsan_epilogue lib/ubsan.c:217 [inline]
__ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
do_check_main kernel/bpf/verifier.c:20591 [inline]
bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
__sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
__do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
__se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
__x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
do_syscall_64+0xfb/0x240
entry_SYSCALL_64_after_hwframe+0x6d/0x75
RIP: 0033:0x7f8416194629
Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
</TASK>
---[ end trace ]---
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
If the report is already addressed, let syzbot know by replying with:
#syz fix: exact-commit-title
If you want syzbot to run the reproducer, reply with:
#syz test: git://repo/address.git branch-or-commit-hash
If you attach or paste a git patch, syzbot will apply it before testing.
If you want to overwrite report's subsystems, reply with:
#syz set subsystems: new-subsystem
(See the list of subsystem names on the web dashboard)
If the report is a duplicate of another one, reply with:
#syz dup: exact-subject-of-another-report
If you want to undo deduplication, reply with:
#syz undup
^ permalink raw reply [flat|nested] 8+ messages in thread
* stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-19 18:12 [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized syzbot
@ 2024-03-21 7:33 ` Alexei Starovoitov
2024-03-21 14:07 ` Andrei Matei
2024-03-24 0:50 ` Andrei Matei
0 siblings, 2 replies; 8+ messages in thread
From: Alexei Starovoitov @ 2024-03-21 7:33 UTC (permalink / raw
To: Andrei Matei
Cc: Andrii Nakryiko, Alexei Starovoitov, bpf, Daniel Borkmann, Eddy Z,
Hao Luo, John Fastabend, Jiri Olsa, KP Singh, LKML,
Martin KaFai Lau, Network Development, Stanislav Fomichev,
Song Liu, syzkaller-bugs, Yonghong Song
Hi Andrei,
looks like the refactoring of stack access introduced a bug.
See the reproducer below.
positive offsets are not caught by check_stack_access_within_bounds().
So both slot and spi become negative and access
stack[spi].slot_type[slot % BPF_REG_SIZE]
returns garbage.
On Tue, Mar 19, 2024 at 11:12 AM syzbot
<syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com> wrote:
>
> Hello,
>
> syzbot found the following issue on:
>
> HEAD commit: 0740b6427e90 Merge branch 'bpf-arena-followups'
> git tree: bpf
> console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
> kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
> compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
>
> Downloadable assets:
> disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
> vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
> kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
>
> IMPORTANT: if you fix the issue, please add the following tag to the commit:
> Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
>
> ------------[ cut here ]------------
> UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
> index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
> CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
> Call Trace:
> <TASK>
> __dump_stack lib/dump_stack.c:88 [inline]
> dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
> ubsan_epilogue lib/ubsan.c:217 [inline]
> __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
> check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
> check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
> check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
> do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
> do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
> do_check_main kernel/bpf/verifier.c:20591 [inline]
> bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
> bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
> __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
> __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
> __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
> __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
> do_syscall_64+0xfb/0x240
> entry_SYSCALL_64_after_hwframe+0x6d/0x75
> RIP: 0033:0x7f8416194629
> Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
> RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
> RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
> R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
> R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
> </TASK>
> ---[ end trace ]---
>
>
> ---
> This report is generated by a bot. It may contain errors.
> See https://goo.gl/tpsmEJ for more information about syzbot.
> syzbot engineers can be reached at syzkaller@googlegroups.com.
>
> syzbot will keep track of this issue. See:
> https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
>
> If the report is already addressed, let syzbot know by replying with:
> #syz fix: exact-commit-title
>
> If you want syzbot to run the reproducer, reply with:
> #syz test: git://repo/address.git branch-or-commit-hash
> If you attach or paste a git patch, syzbot will apply it before testing.
>
> If you want to overwrite report's subsystems, reply with:
> #syz set subsystems: new-subsystem
> (See the list of subsystem names on the web dashboard)
>
> If the report is a duplicate of another one, reply with:
> #syz dup: exact-subject-of-another-report
>
> If you want to undo deduplication, reply with:
> #syz undup
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-21 7:33 ` stack access issue. " Alexei Starovoitov
@ 2024-03-21 14:07 ` Andrei Matei
2024-03-24 0:50 ` Andrei Matei
1 sibling, 0 replies; 8+ messages in thread
From: Andrei Matei @ 2024-03-21 14:07 UTC (permalink / raw
To: Alexei Starovoitov
Cc: Andrii Nakryiko, Alexei Starovoitov, bpf, Daniel Borkmann, Eddy Z,
Hao Luo, John Fastabend, Jiri Olsa, KP Singh, LKML,
Martin KaFai Lau, Network Development, Stanislav Fomichev,
Song Liu, syzkaller-bugs, Yonghong Song
Thanks for the report! Will look in a bit.
On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> Hi Andrei,
>
> looks like the refactoring of stack access introduced a bug.
> See the reproducer below.
> positive offsets are not caught by check_stack_access_within_bounds().
> So both slot and spi become negative and access
> stack[spi].slot_type[slot % BPF_REG_SIZE]
> returns garbage.
>
> On Tue, Mar 19, 2024 at 11:12 AM syzbot
> <syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit: 0740b6427e90 Merge branch 'bpf-arena-followups'
> > git tree: bpf
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> > dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
> > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
> >
> > ------------[ cut here ]------------
> > UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
> > index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
> > CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
> > Call Trace:
> > <TASK>
> > __dump_stack lib/dump_stack.c:88 [inline]
> > dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
> > ubsan_epilogue lib/ubsan.c:217 [inline]
> > __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
> > check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
> > check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
> > check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
> > do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
> > do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
> > do_check_main kernel/bpf/verifier.c:20591 [inline]
> > bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
> > bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
> > __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
> > __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
> > __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
> > __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
> > do_syscall_64+0xfb/0x240
> > entry_SYSCALL_64_after_hwframe+0x6d/0x75
> > RIP: 0033:0x7f8416194629
> > Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> > RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> > RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
> > RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
> > RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
> > R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
> > R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
> > </TASK>
> > ---[ end trace ]---
> >
> >
> > ---
> > This report is generated by a bot. It may contain errors.
> > See https://goo.gl/tpsmEJ for more information about syzbot.
> > syzbot engineers can be reached at syzkaller@googlegroups.com.
> >
> > syzbot will keep track of this issue. See:
> > https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> >
> > If the report is already addressed, let syzbot know by replying with:
> > #syz fix: exact-commit-title
> >
> > If you want syzbot to run the reproducer, reply with:
> > #syz test: git://repo/address.git branch-or-commit-hash
> > If you attach or paste a git patch, syzbot will apply it before testing.
> >
> > If you want to overwrite report's subsystems, reply with:
> > #syz set subsystems: new-subsystem
> > (See the list of subsystem names on the web dashboard)
> >
> > If the report is a duplicate of another one, reply with:
> > #syz dup: exact-subject-of-another-report
> >
> > If you want to undo deduplication, reply with:
> > #syz undup
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-21 7:33 ` stack access issue. " Alexei Starovoitov
2024-03-21 14:07 ` Andrei Matei
@ 2024-03-24 0:50 ` Andrei Matei
2024-03-24 0:52 ` Alexei Starovoitov
1 sibling, 1 reply; 8+ messages in thread
From: Andrei Matei @ 2024-03-24 0:50 UTC (permalink / raw
To: Alexei Starovoitov, eadavis
Cc: Andrii Nakryiko, Alexei Starovoitov, bpf, Daniel Borkmann, Eddy Z,
Hao Luo, John Fastabend, Jiri Olsa, KP Singh, LKML,
Martin KaFai Lau, Network Development, Stanislav Fomichev,
Song Liu, syzkaller-bugs, Yonghong Song
+ Edward
On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> Hi Andrei,
>
> looks like the refactoring of stack access introduced a bug.
> See the reproducer below.
> positive offsets are not caught by check_stack_access_within_bounds().
check_stack_access_within_bounds() tries to catch positive offsets;
It does: [1]
err = check_stack_slot_within_bounds(env, min_off, state, type);
if (!err && max_off > 0)
err = -EINVAL; /* out of stack access into non-negative offsets */
Notice the max_off > 0 in there.
And we have various tests that seem to check that positive offsets are
rejected. Do you know what the bug is?
I'm thinking maybe there's some overflow going on, except that UBSAN
reported an index of -1 as being the problem.
Edward, I see that you've been tickling the robot trying to narrow the issue;
perhaps you've figured it out?
If the bug is not immediately apparent to anyone, I would really appreciate a
bit of tutoring around how to reproduce and get verifier logs. I have tried a
bunch of cases of constant- and variable-offset accesses, and couldn't repro. I
can run syzkaller's repro on its own vm image, and indeed it crashes. But I'm
not sure how to get verifier logs out of the C reproducer. Alternatively, I'm
not sure how to figure out the actual BPF program corresponding to the "syz
repro" in [2] and turn it into a test_progs test. How do you guys do it?
Thanks a lot!
[1] https://github.com/torvalds/linux/blob/70293240c5ce675a67bfc48f419b093023b862b3/kernel/bpf/verifier.c#L6695
[2] https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> So both slot and spi become negative and access
> stack[spi].slot_type[slot % BPF_REG_SIZE]
> returns garbage.
>
> On Tue, Mar 19, 2024 at 11:12 AM syzbot
> <syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com> wrote:
> >
> > Hello,
> >
> > syzbot found the following issue on:
> >
> > HEAD commit: 0740b6427e90 Merge branch 'bpf-arena-followups'
> > git tree: bpf
> > console+strace: https://syzkaller.appspot.com/x/log.txt?x=12fed769180000
> > kernel config: https://syzkaller.appspot.com/x/.config?x=6fb1be60a193d440
> > dashboard link: https://syzkaller.appspot.com/bug?extid=33f4297b5f927648741a
> > compiler: Debian clang version 15.0.6, GNU ld (GNU Binutils for Debian) 2.40
> > syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1763a479180000
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
> >
> > Downloadable assets:
> > disk image: https://storage.googleapis.com/syzbot-assets/c9e6e9f97566/disk-0740b642.raw.xz
> > vmlinux: https://storage.googleapis.com/syzbot-assets/78476a588b62/vmlinux-0740b642.xz
> > kernel image: https://storage.googleapis.com/syzbot-assets/50cd6fab9ead/bzImage-0740b642.xz
> >
> > IMPORTANT: if you fix the issue, please add the following tag to the commit:
> > Reported-by: syzbot+33f4297b5f927648741a@syzkaller.appspotmail.com
> >
> > ------------[ cut here ]------------
> > UBSAN: array-index-out-of-bounds in kernel/bpf/verifier.c:7190:12
> > index -1 is out of range for type 'u8[8]' (aka 'unsigned char[8]')
> > CPU: 0 PID: 5071 Comm: syz-executor474 Not tainted 6.8.0-syzkaller-05226-g0740b6427e90 #0
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 02/29/2024
> > Call Trace:
> > <TASK>
> > __dump_stack lib/dump_stack.c:88 [inline]
> > dump_stack_lvl+0x1e7/0x2e0 lib/dump_stack.c:106
> > ubsan_epilogue lib/ubsan.c:217 [inline]
> > __ubsan_handle_out_of_bounds+0x121/0x150 lib/ubsan.c:415
> > check_stack_range_initialized+0x1668/0x19a0 kernel/bpf/verifier.c:7190
> > check_helper_mem_access+0x2eb/0xfa0 kernel/bpf/verifier.c:7294
> > check_helper_call+0x263c/0x7220 kernel/bpf/verifier.c:10252
> > do_check+0x9e29/0x10530 kernel/bpf/verifier.c:17801
> > do_check_common+0x14bd/0x1dd0 kernel/bpf/verifier.c:20500
> > do_check_main kernel/bpf/verifier.c:20591 [inline]
> > bpf_check+0x136ab/0x19010 kernel/bpf/verifier.c:21261
> > bpf_prog_load+0x1667/0x20f0 kernel/bpf/syscall.c:2895
> > __sys_bpf+0x4ee/0x810 kernel/bpf/syscall.c:5631
> > __do_sys_bpf kernel/bpf/syscall.c:5738 [inline]
> > __se_sys_bpf kernel/bpf/syscall.c:5736 [inline]
> > __x64_sys_bpf+0x7c/0x90 kernel/bpf/syscall.c:5736
> > do_syscall_64+0xfb/0x240
> > entry_SYSCALL_64_after_hwframe+0x6d/0x75
> > RIP: 0033:0x7f8416194629
> > Code: 48 83 c4 28 c3 e8 37 17 00 00 0f 1f 80 00 00 00 00 48 89 f8 48 89 f7 48 89 d6 48 89 ca 4d 89 c2 4d 89 c8 4c 8b 4c 24 08 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 c7 c1 b8 ff ff ff f7 d8 64 89 01 48
> > RSP: 002b:00007ffdc6f0fdb8 EFLAGS: 00000246 ORIG_RAX: 0000000000000141
> > RAX: ffffffffffffffda RBX: 00007ffdc6f0ff88 RCX: 00007f8416194629
> > RDX: 0000000000000090 RSI: 00000000200000c0 RDI: 0000000000000005
> > RBP: 00007f8416207610 R08: 0000000000000000 R09: 00007ffdc6f0ff88
> > R10: 00000000fffffff8 R11: 0000000000000246 R12: 0000000000000001
> > R13: 00007ffdc6f0ff78 R14: 0000000000000001 R15: 0000000000000001
> > </TASK>
> > ---[ end trace ]---
> >
> >
> > ---
> > This report is generated by a bot. It may contain errors.
> > See https://goo.gl/tpsmEJ for more information about syzbot.
> > syzbot engineers can be reached at syzkaller@googlegroups.com.
> >
> > syzbot will keep track of this issue. See:
> > https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
> >
> > If the report is already addressed, let syzbot know by replying with:
> > #syz fix: exact-commit-title
> >
> > If you want syzbot to run the reproducer, reply with:
> > #syz test: git://repo/address.git branch-or-commit-hash
> > If you attach or paste a git patch, syzbot will apply it before testing.
> >
> > If you want to overwrite report's subsystems, reply with:
> > #syz set subsystems: new-subsystem
> > (See the list of subsystem names on the web dashboard)
> >
> > If the report is a duplicate of another one, reply with:
> > #syz dup: exact-subject-of-another-report
> >
> > If you want to undo deduplication, reply with:
> > #syz undup
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-24 0:50 ` Andrei Matei
@ 2024-03-24 0:52 ` Alexei Starovoitov
2024-03-24 2:12 ` Andrei Matei
0 siblings, 1 reply; 8+ messages in thread
From: Alexei Starovoitov @ 2024-03-24 0:52 UTC (permalink / raw
To: Andrei Matei
Cc: Edward Adam Davis, Andrii Nakryiko, Alexei Starovoitov, bpf,
Daniel Borkmann, Eddy Z, Hao Luo, John Fastabend, Jiri Olsa,
KP Singh, LKML, Martin KaFai Lau, Network Development,
Stanislav Fomichev, Song Liu, syzkaller-bugs, Yonghong Song
On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
>
> + Edward
>
> On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > Hi Andrei,
> >
> > looks like the refactoring of stack access introduced a bug.
> > See the reproducer below.
> > positive offsets are not caught by check_stack_access_within_bounds().
>
> check_stack_access_within_bounds() tries to catch positive offsets;
> It does: [1]
>
> err = check_stack_slot_within_bounds(env, min_off, state, type);
> if (!err && max_off > 0)
> err = -EINVAL; /* out of stack access into non-negative offsets */
>
> Notice the max_off > 0 in there.
> And we have various tests that seem to check that positive offsets are
> rejected. Do you know what the bug is?
> I'm thinking maybe there's some overflow going on, except that UBSAN
> reported an index of -1 as being the problem.
>
> Edward, I see that you've been tickling the robot trying to narrow the issue;
> perhaps you've figured it out?
>
> If the bug is not immediately apparent to anyone, I would really appreciate a
> bit of tutoring around how to reproduce and get verifier logs.
The repro is right there in the email I forwarded:
> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-24 0:52 ` Alexei Starovoitov
@ 2024-03-24 2:12 ` Andrei Matei
2024-03-24 2:55 ` Alexei Starovoitov
0 siblings, 1 reply; 8+ messages in thread
From: Andrei Matei @ 2024-03-24 2:12 UTC (permalink / raw
To: Alexei Starovoitov
Cc: Edward Adam Davis, Andrii Nakryiko, Alexei Starovoitov, bpf,
Daniel Borkmann, Eddy Z, Hao Luo, John Fastabend, Jiri Olsa,
KP Singh, LKML, Martin KaFai Lau, Network Development,
Stanislav Fomichev, Song Liu, syzkaller-bugs, Yonghong Song
On Sat, Mar 23, 2024 at 8:52 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> >
> > + Edward
> >
> > On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > Hi Andrei,
> > >
> > > looks like the refactoring of stack access introduced a bug.
> > > See the reproducer below.
> > > positive offsets are not caught by check_stack_access_within_bounds().
> >
> > check_stack_access_within_bounds() tries to catch positive offsets;
> > It does: [1]
> >
> > err = check_stack_slot_within_bounds(env, min_off, state, type);
> > if (!err && max_off > 0)
> > err = -EINVAL; /* out of stack access into non-negative offsets */
> >
> > Notice the max_off > 0 in there.
> > And we have various tests that seem to check that positive offsets are
> > rejected. Do you know what the bug is?
> > I'm thinking maybe there's some overflow going on, except that UBSAN
> > reported an index of -1 as being the problem.
> >
> > Edward, I see that you've been tickling the robot trying to narrow the issue;
> > perhaps you've figured it out?
> >
> > If the bug is not immediately apparent to anyone, I would really appreciate a
> > bit of tutoring around how to reproduce and get verifier logs.
>
> The repro is right there in the email I forwarded:
>
> > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
I understand, but how does one go from this to either BPF assembly,
or to running it in such a way that you also get verifier logs?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-24 2:12 ` Andrei Matei
@ 2024-03-24 2:55 ` Alexei Starovoitov
2024-03-26 2:48 ` Andrei Matei
0 siblings, 1 reply; 8+ messages in thread
From: Alexei Starovoitov @ 2024-03-24 2:55 UTC (permalink / raw
To: Andrei Matei
Cc: Edward Adam Davis, Andrii Nakryiko, Alexei Starovoitov, bpf,
Daniel Borkmann, Eddy Z, Hao Luo, John Fastabend, Jiri Olsa,
KP Singh, LKML, Martin KaFai Lau, Network Development,
Stanislav Fomichev, Song Liu, syzkaller-bugs, Yonghong Song
On Sat, Mar 23, 2024 at 7:12 PM Andrei Matei <andreimatei1@gmail.com> wrote:
>
> On Sat, Mar 23, 2024 at 8:52 PM Alexei Starovoitov
> <alexei.starovoitov@gmail.com> wrote:
> >
> > On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> > >
> > > + Edward
> > >
> > > On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> > > <alexei.starovoitov@gmail.com> wrote:
> > > >
> > > > Hi Andrei,
> > > >
> > > > looks like the refactoring of stack access introduced a bug.
> > > > See the reproducer below.
> > > > positive offsets are not caught by check_stack_access_within_bounds().
> > >
> > > check_stack_access_within_bounds() tries to catch positive offsets;
> > > It does: [1]
> > >
> > > err = check_stack_slot_within_bounds(env, min_off, state, type);
> > > if (!err && max_off > 0)
> > > err = -EINVAL; /* out of stack access into non-negative offsets */
> > >
> > > Notice the max_off > 0 in there.
> > > And we have various tests that seem to check that positive offsets are
> > > rejected. Do you know what the bug is?
> > > I'm thinking maybe there's some overflow going on, except that UBSAN
> > > reported an index of -1 as being the problem.
> > >
> > > Edward, I see that you've been tickling the robot trying to narrow the issue;
> > > perhaps you've figured it out?
> > >
> > > If the bug is not immediately apparent to anyone, I would really appreciate a
> > > bit of tutoring around how to reproduce and get verifier logs.
> >
> > The repro is right there in the email I forwarded:
> >
> > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
>
> I understand, but how does one go from this to either BPF assembly,
> or to running it in such a way that you also get verifier logs?
Adding logs to repro.c is too hard, but you can
hack the kernel with printk-s.
Like the following:
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index de7813947981..d158b83ed16c 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -7179,6 +7179,7 @@ static int check_stack_range_initialized(
return -EFAULT;
}
+ printk("slot %d %d spi %d\n", slot, slot % BPF_REG_SIZE, spi);
stype = &state->stack[spi].slot_type[slot % BPF_REG_SIZE];
shows that spi and slot get negative: -1, -2, ...
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: stack access issue. Re: [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized
2024-03-24 2:55 ` Alexei Starovoitov
@ 2024-03-26 2:48 ` Andrei Matei
0 siblings, 0 replies; 8+ messages in thread
From: Andrei Matei @ 2024-03-26 2:48 UTC (permalink / raw
To: Alexei Starovoitov
Cc: Edward Adam Davis, Andrii Nakryiko, Alexei Starovoitov, bpf,
Daniel Borkmann, Eddy Z, Hao Luo, John Fastabend, Jiri Olsa,
KP Singh, LKML, Martin KaFai Lau, Network Development,
Stanislav Fomichev, Song Liu, syzkaller-bugs, Yonghong Song
Fixing in https://lore.kernel.org/bpf/20240324230323.1097685-1-andreimatei1@gmail.com/
FWIW, I managed to decode the BPF program that syzkaller used:
0: (18) r0 = 0x0
2: (18) r1 = map[id:4]
4: (b7) r8 = 0
5: (7b) *(u64 *)(r10 -8) = r8
6: (bf) r2 = r10
7: (07) r2 += -8
8: (b7) r3 = 8
9: (b7) r4 = 0
10: (85) call bloom_map_peek_elem#322320
11: (95) exit
Where the map is a bloom filter (as Alexei somehow already knew on the patch
thread) with a humongous value size.
4: type 30 flags 0x0
key 0B value 2147483649B max_entries 255 memlock 720B
On Sat, Mar 23, 2024 at 10:55 PM Alexei Starovoitov
<alexei.starovoitov@gmail.com> wrote:
>
> On Sat, Mar 23, 2024 at 7:12 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> >
> > On Sat, Mar 23, 2024 at 8:52 PM Alexei Starovoitov
> > <alexei.starovoitov@gmail.com> wrote:
> > >
> > > On Sat, Mar 23, 2024 at 5:50 PM Andrei Matei <andreimatei1@gmail.com> wrote:
> > > >
> > > > + Edward
> > > >
> > > > On Thu, Mar 21, 2024 at 3:33 AM Alexei Starovoitov
> > > > <alexei.starovoitov@gmail.com> wrote:
> > > > >
> > > > > Hi Andrei,
> > > > >
> > > > > looks like the refactoring of stack access introduced a bug.
> > > > > See the reproducer below.
> > > > > positive offsets are not caught by check_stack_access_within_bounds().
> > > >
> > > > check_stack_access_within_bounds() tries to catch positive offsets;
> > > > It does: [1]
> > > >
> > > > err = check_stack_slot_within_bounds(env, min_off, state, type);
> > > > if (!err && max_off > 0)
> > > > err = -EINVAL; /* out of stack access into non-negative offsets */
> > > >
> > > > Notice the max_off > 0 in there.
> > > > And we have various tests that seem to check that positive offsets are
> > > > rejected. Do you know what the bug is?
> > > > I'm thinking maybe there's some overflow going on, except that UBSAN
> > > > reported an index of -1 as being the problem.
> > > >
> > > > Edward, I see that you've been tickling the robot trying to narrow the issue;
> > > > perhaps you've figured it out?
> > > >
> > > > If the bug is not immediately apparent to anyone, I would really appreciate a
> > > > bit of tutoring around how to reproduce and get verifier logs.
> > >
> > > The repro is right there in the email I forwarded:
> > >
> > > > C reproducer: https://syzkaller.appspot.com/x/repro.c?x=15c38711180000
> >
> > I understand, but how does one go from this to either BPF assembly,
> > or to running it in such a way that you also get verifier logs?
>
> Adding logs to repro.c is too hard, but you can
> hack the kernel with printk-s.
>
> Like the following:
>
> diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
> index de7813947981..d158b83ed16c 100644
> --- a/kernel/bpf/verifier.c
> +++ b/kernel/bpf/verifier.c
> @@ -7179,6 +7179,7 @@ static int check_stack_range_initialized(
> return -EFAULT;
> }
>
> + printk("slot %d %d spi %d\n", slot, slot % BPF_REG_SIZE, spi);
> stype = &state->stack[spi].slot_type[slot % BPF_REG_SIZE];
>
>
> shows that spi and slot get negative: -1, -2, ...
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2024-03-26 2:48 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-03-19 18:12 [syzbot] [bpf?] UBSAN: array-index-out-of-bounds in check_stack_range_initialized syzbot
2024-03-21 7:33 ` stack access issue. " Alexei Starovoitov
2024-03-21 14:07 ` Andrei Matei
2024-03-24 0:50 ` Andrei Matei
2024-03-24 0:52 ` Alexei Starovoitov
2024-03-24 2:12 ` Andrei Matei
2024-03-24 2:55 ` Alexei Starovoitov
2024-03-26 2:48 ` Andrei Matei
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).