* [Bug 208907] New: [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on
@ 2020-08-14 18:43 bugzilla-daemon
2020-08-14 19:19 ` [Bug 208907] " bugzilla-daemon
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: bugzilla-daemon @ 2020-08-14 18:43 UTC (permalink / raw
To: linux-xfs
https://bugzilla.kernel.org/show_bug.cgi?id=208907
Bug ID: 208907
Summary: [fstests generic/074 on xfs]: 5.7.10 fails with a hung
task on
Product: File System
Version: 2.5
Kernel Version: 5.7.10
Hardware: x86-64
OS: Linux
Tree: Mainline
Status: NEW
Severity: normal
Priority: P1
Component: XFS
Assignee: filesystem_xfs@kernel-bugs.kernel.org
Reporter: mcgrof@kernel.org
Regression: No
Should be extremely easy to reproduce in 5 commands with kdevops [0], leave
everything with the defaults, and then just enable fstests.
[0] https://github.com/mcgrof/kdevops
make bringup
make fstests
Just ssh to kdevops-xfs and run:
cd /var/lib/xfstests/
./gendisks.sh -m
./check generic/074
Aug 14 18:27:34 kdevops-xfs-dev kernel: XFS (loop16): Mounting V5 Filesystem
Aug 14 18:27:34 kdevops-xfs-dev kernel: XFS (loop16): Ending clean mount
Aug 14 18:27:34 kdevops-xfs-dev kernel: xfs filesystem being mounted at
/media/test supports timestamps until 2038 (0x7fffffff)
Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
aborting
Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: Abort status: 0x4001
Aug 14 18:28:47 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
reset controller
Aug 14 18:28:54 kdevops-xfs-dev kernel: sched: RT throttling activated
Aug 14 18:31:12 kdevops-xfs-dev kernel: INFO: task xfsaild/nvme1n1:289 blocked
for more than 120 seconds.
Aug 14 18:31:12 kdevops-xfs-dev kernel: Not tainted 5.7.0-2-amd64 #1
Debian 5.7.10-1
Aug 14 18:31:12 kdevops-xfs-dev kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 14 18:31:12 kdevops-xfs-dev kernel: xfsaild/nvme1n1 D 0 289 2
0x80004000
Aug 14 18:31:12 kdevops-xfs-dev kernel: Call Trace:
Aug 14 18:31:12 kdevops-xfs-dev kernel: __schedule+0x2da/0x770
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? xlog_cil_force_lsn+0xc0/0x220 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: schedule+0x4a/0xb0
Aug 14 18:31:12 kdevops-xfs-dev kernel: xlog_wait_on_iclog+0x113/0x130 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? wake_up_q+0xa0/0xa0
Aug 14 18:31:12 kdevops-xfs-dev kernel: xfsaild+0x1bd/0x810 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? __switch_to+0x80/0x3c0
Aug 14 18:31:12 kdevops-xfs-dev kernel: kthread+0xf9/0x130
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? xfs_trans_ail_cursor_first+0x80/0x80
[xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? kthread_park+0x90/0x90
Aug 14 18:31:12 kdevops-xfs-dev kernel: ret_from_fork+0x35/0x40
Aug 14 18:31:12 kdevops-xfs-dev kernel: INFO: task loop16:912 blocked for more
than 120 seconds.
Aug 14 18:31:12 kdevops-xfs-dev kernel: Not tainted 5.7.0-2-amd64 #1
Debian 5.7.10-1
Aug 14 18:31:12 kdevops-xfs-dev kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 14 18:31:12 kdevops-xfs-dev kernel: loop16 D 0 912 2
0x80004000
Aug 14 18:31:12 kdevops-xfs-dev kernel: Call Trace:
Aug 14 18:31:12 kdevops-xfs-dev kernel: __schedule+0x2da/0x770
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? xlog_cil_force_lsn+0xc0/0x220 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: schedule+0x4a/0xb0
Aug 14 18:31:12 kdevops-xfs-dev kernel: xlog_wait_on_iclog+0x113/0x130 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? wake_up_q+0xa0/0xa0
Aug 14 18:31:12 kdevops-xfs-dev kernel: xfsaild+0x1bd/0x810 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? __switch_to+0x80/0x3c0
Aug 14 18:31:12 kdevops-xfs-dev kernel: kthread+0xf9/0x130
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? xfs_trans_ail_cursor_first+0x80/0x80
[xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? kthread_park+0x90/0x90
Aug 14 18:31:12 kdevops-xfs-dev kernel: ret_from_fork+0x35/0x40
Aug 14 18:31:12 kdevops-xfs-dev kernel: INFO: task loop16:912 blocked for more
than 120 seconds.
Aug 14 18:31:12 kdevops-xfs-dev kernel: Not tainted 5.7.0-2-amd64 #1
Debian 5.7.10-1
Aug 14 18:31:12 kdevops-xfs-dev kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 14 18:31:12 kdevops-xfs-dev kernel: loop16 D 0 912 2
0x80004000
Aug 14 18:31:12 kdevops-xfs-dev kernel: Call Trace:
Aug 14 18:31:12 kdevops-xfs-dev kernel: __schedule+0x2da/0x770
Aug 14 18:31:12 kdevops-xfs-dev kernel: schedule+0x4a/0xb0
Aug 14 18:31:12 kdevops-xfs-dev kernel: xlog_wait_on_iclog+0x113/0x130 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? wake_up_q+0xa0/0xa0
Aug 14 18:31:12 kdevops-xfs-dev kernel: __xfs_log_force_lsn+0x10a/0x1d0 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? xfs_file_fsync+0x1f4/0x230 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: xfs_log_force_lsn+0x91/0x120 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: xfs_file_fsync+0x1f4/0x230 [xfs]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? __switch_to_asm+0x34/0x70
Aug 14 18:31:12 kdevops-xfs-dev kernel: loop_queue_work+0x47d/0xa50 [loop]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? __switch_to+0x80/0x3c0
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? __schedule+0x2e2/0x770
Aug 14 18:31:12 kdevops-xfs-dev kernel: kthread_worker_fn+0x73/0x1d0
Aug 14 18:31:12 kdevops-xfs-dev kernel: kthread+0xf9/0x130
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? loop_info64_to_compat+0x220/0x220
[loop]
Aug 14 18:31:12 kdevops-xfs-dev kernel: ? kthread_park+0x90/0x90
Aug 14 18:31:12 kdevops-xfs-dev kernel: ret_from_fork+0x35/0x40
Aug 14 18:31:12 kdevops-xfs-dev kernel: INFO: task umount:2212 blocked for more
than 120 seconds.
Aug 14 18:31:12 kdevops-xfs-dev kernel: Not tainted 5.7.0-2-amd64 #1
Debian 5.7.10-1
Aug 14 18:31:12 kdevops-xfs-dev kernel: "echo 0 >
/proc/sys/kernel/hung_task_timeout_secs" disables this message.
Aug 14 18:31:12 kdevops-xfs-dev kernel: umount D 0 2212 2208
0x00004000
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Bug 208907] [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on
2020-08-14 18:43 [Bug 208907] New: [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on bugzilla-daemon
@ 2020-08-14 19:19 ` bugzilla-daemon
2020-08-14 22:41 ` [Bug 208907] New: " Dave Chinner
2020-08-14 22:41 ` [Bug 208907] " bugzilla-daemon
2 siblings, 0 replies; 4+ messages in thread
From: bugzilla-daemon @ 2020-08-14 19:19 UTC (permalink / raw
To: linux-xfs
https://bugzilla.kernel.org/show_bug.cgi?id=208907
Eric Sandeen (sandeen@redhat.com) changed:
What |Removed |Added
----------------------------------------------------------------------------
CC| |sandeen@redhat.com
--- Comment #1 from Eric Sandeen (sandeen@redhat.com) ---
This seems relevant, no?
> Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
> aborting
> Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: Abort status: 0x4001
> Aug 14 18:28:47 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
> reset controller
then 2.5 minutes later,
> Aug 14 18:31:12 kdevops-xfs-dev kernel: INFO: task xfsaild/nvme1n1:289
> blocked for more than 120 seconds.
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [Bug 208907] New: [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on
2020-08-14 18:43 [Bug 208907] New: [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on bugzilla-daemon
2020-08-14 19:19 ` [Bug 208907] " bugzilla-daemon
@ 2020-08-14 22:41 ` Dave Chinner
2020-08-14 22:41 ` [Bug 208907] " bugzilla-daemon
2 siblings, 0 replies; 4+ messages in thread
From: Dave Chinner @ 2020-08-14 22:41 UTC (permalink / raw
To: bugzilla-daemon; +Cc: linux-xfs
On Fri, Aug 14, 2020 at 06:43:18PM +0000, bugzilla-daemon@bugzilla.kernel.org wrote:
> Just ssh to kdevops-xfs and run:
>
> cd /var/lib/xfstests/
> ./gendisks.sh -m
> ./check generic/074
>
> Aug 14 18:27:34 kdevops-xfs-dev kernel: XFS (loop16): Mounting V5 Filesystem
> Aug 14 18:27:34 kdevops-xfs-dev kernel: XFS (loop16): Ending clean mount
> Aug 14 18:27:34 kdevops-xfs-dev kernel: xfs filesystem being mounted at
> /media/test supports timestamps until 2038 (0x7fffffff)
> Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
> aborting
> Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: Abort status: 0x4001
> Aug 14 18:28:47 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
> reset controller
Hardware lost an IO. I'm guessing the error handling that reset the
controller failed to error out the bio the lost IO belonged to, so
XFS has hung waiting for it...
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
^ permalink raw reply [flat|nested] 4+ messages in thread
* [Bug 208907] [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on
2020-08-14 18:43 [Bug 208907] New: [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on bugzilla-daemon
2020-08-14 19:19 ` [Bug 208907] " bugzilla-daemon
2020-08-14 22:41 ` [Bug 208907] New: " Dave Chinner
@ 2020-08-14 22:41 ` bugzilla-daemon
2 siblings, 0 replies; 4+ messages in thread
From: bugzilla-daemon @ 2020-08-14 22:41 UTC (permalink / raw
To: linux-xfs
https://bugzilla.kernel.org/show_bug.cgi?id=208907
--- Comment #2 from Dave Chinner (david@fromorbit.com) ---
On Fri, Aug 14, 2020 at 06:43:18PM +0000, bugzilla-daemon@bugzilla.kernel.org
wrote:
> Just ssh to kdevops-xfs and run:
>
> cd /var/lib/xfstests/
> ./gendisks.sh -m
> ./check generic/074
>
> Aug 14 18:27:34 kdevops-xfs-dev kernel: XFS (loop16): Mounting V5 Filesystem
> Aug 14 18:27:34 kdevops-xfs-dev kernel: XFS (loop16): Ending clean mount
> Aug 14 18:27:34 kdevops-xfs-dev kernel: xfs filesystem being mounted at
> /media/test supports timestamps until 2038 (0x7fffffff)
> Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
> aborting
> Aug 14 18:28:16 kdevops-xfs-dev kernel: nvme nvme1: Abort status: 0x4001
> Aug 14 18:28:47 kdevops-xfs-dev kernel: nvme nvme1: I/O 128 QID 2 timeout,
> reset controller
Hardware lost an IO. I'm guessing the error handling that reset the
controller failed to error out the bio the lost IO belonged to, so
XFS has hung waiting for it...
Cheers,
Dave.
--
You are receiving this mail because:
You are watching the assignee of the bug.
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2020-08-14 22:41 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2020-08-14 18:43 [Bug 208907] New: [fstests generic/074 on xfs]: 5.7.10 fails with a hung task on bugzilla-daemon
2020-08-14 19:19 ` [Bug 208907] " bugzilla-daemon
2020-08-14 22:41 ` [Bug 208907] New: " Dave Chinner
2020-08-14 22:41 ` [Bug 208907] " bugzilla-daemon
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.