* qemu io test: spurious failures of 030
@ 2021-06-10 7:54 Christian Borntraeger
2021-06-10 12:50 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 2+ messages in thread
From: Christian Borntraeger @ 2021-06-10 7:54 UTC (permalink / raw)
To: qemu block, qemu-s390x, qemu-devel
Folks,
I have CI on s390 on qemu/master for each commit. And from time to time I do get spurious failures of 030. (always 030 and nothing else).
I have a hard time reproducing this manually so I cannot debug this at the moment. Has anyone seen this as well?
030 ... [09:34:06] ... 030 fail [09:34:06] [09:34:09] 3.2s failed, exit status 1
--- tests/qemu-iotests/030.out
+++ 030.out.bad
@@ -1,5 +1,45 @@
-...........................
+.............WARNING:qemu.machine.machine:qemu received signal 11; command: "build/tests/qemu-iotests/../../qemu-system-s390x -display none -vga none -chardev socket,id=mon,path=/tmp/tmpt3vg8hqd/qemu-2816608-monitor.sock -mon chardev=mon,mode=control -qtest unix:path=/tmp/tmpt3vg8hqd/qemu-2816608-qtest.sock -accel qtest -nodefaults -display none -accel qtest -drive if=virtio,id=drive0,file=/tmp/devel-iotests-9771/img-8.img,format=qcow2,cache=writeback,aio=threads,backing.backing.backing.backing.backing.backing.backing.backing.node-name=node0,backing.backing.backing.backing.backing.backing.backing.node-name=node1,backing.backing.backing.backing.backing.backing.node-name=node2,backing.backing.backing.backing.backing.node-name=node3,backing.backing.backing.backing.node-name=node4,backing.backing.backing.node-name=node5,backing.backing.node-name=node6,backing.node-name=node7,node-name=node8"
+EE.............
+======================================================================
+ERROR: test_stream_parallel (__main__.TestParallelOps)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "tests/qemu-iotests/030", line 260, in test_stream_parallel
+ for event in self.vm.get_qmp_events(wait=True):
+ File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 597, in get_qmp_events
+ events = self._qmp.get_events(wait=wait)
+ File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 338, in get_events
+ self.__get_events(wait)
+ File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 207, in __get_events
+ raise QMPConnectError("Error while reading from socket")
+qemu.qmp.QMPConnectError: Error while reading from socket
+
+======================================================================
+ERROR: test_stream_parallel (__main__.TestParallelOps)
+----------------------------------------------------------------------
+Traceback (most recent call last):
+ File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 489, in _do_shutdown
+ self._soft_shutdown(timeout, has_quit)
+ File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 469, in _soft_shutdown
+ self._qmp.cmd('quit')
+ File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 288, in cmd
+ return self.cmd_obj(qmp_cmd)
+ File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 266, in cmd_obj
+ self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8'))
+BrokenPipeError: [Errno 32] Broken pipe
+
+The above exception was the direct cause of the following exception:
+
+Traceback (most recent call last):
+ File "tests/qemu-iotests/030", line 227, in tearDown
+ self.vm.shutdown()
+ File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 519, in shutdown
+ self._do_shutdown(timeout, has_quit)
+ File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 492, in _do_shutdown
+ raise AbnormalShutdown("Could not perform graceful shutdown") \
+qemu.machine.machine.AbnormalShutdown: Could not perform graceful shutdown
+
----------------------------------------------------------------------
Ran 27 tests
-OK
+FAILED (errors=2)
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: qemu io test: spurious failures of 030
2021-06-10 7:54 qemu io test: spurious failures of 030 Christian Borntraeger
@ 2021-06-10 12:50 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 2+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2021-06-10 12:50 UTC (permalink / raw)
To: Christian Borntraeger, qemu block, qemu-s390x, qemu-devel; +Cc: Claudio Fontana
Hi!
030 is known to crash sometimes
The only thing we have (as far as I know) for now is my outdated "[PATCH RFC 0/5] Fix accidental crash in iotest 30" (6 moths ago, time flies fast :\
https://patchew.org/QEMU/20201120161622.1537-1-vsementsov@virtuozzo.com/
I hope one day I'll return to this thing.. But everybody welcome to get ahead of me :)
Note: we dropped iotest 30 from auto group in
commit e2c5093c993ef646e4e28f7aa78429853bcc06ac
Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Date: Fri Feb 5 14:10:21 2021 +0300
iotests: 30: drop from auto group (and effectively from make check)
And original commit message (look at <20210205111021.715240-1-vsementsov@virtuozzo.com> in list) said
So it seems reasonable to drop test from auto group at least until we
merge "block: update graph permissions update"
(and it also showed that test still crash on these series, but much less often)
Now this series is merged. And it seems to me that really, I see crashes in iotest 30 rarely. But they still happen sometimes.
10.06.2021 10:54, Christian Borntraeger wrote:
> Folks,
>
> I have CI on s390 on qemu/master for each commit. And from time to time I do get spurious failures of 030. (always 030 and nothing else).
> I have a hard time reproducing this manually so I cannot debug this at the moment. Has anyone seen this as well?
>
>
> 030 ... [09:34:06] ... 030 fail [09:34:06] [09:34:09] 3.2s failed, exit status 1
> --- tests/qemu-iotests/030.out
> +++ 030.out.bad
> @@ -1,5 +1,45 @@
> -...........................
> +.............WARNING:qemu.machine.machine:qemu received signal 11; command: "build/tests/qemu-iotests/../../qemu-system-s390x -display none -vga none -chardev socket,id=mon,path=/tmp/tmpt3vg8hqd/qemu-2816608-monitor.sock -mon chardev=mon,mode=control -qtest unix:path=/tmp/tmpt3vg8hqd/qemu-2816608-qtest.sock -accel qtest -nodefaults -display none -accel qtest -drive if=virtio,id=drive0,file=/tmp/devel-iotests-9771/img-8.img,format=qcow2,cache=writeback,aio=threads,backing.backing.backing.backing.backing.backing.backing.backing.node-name=node0,backing.backing.backing.backing.backing.backing.backing.node-name=node1,backing.backing.backing.backing.backing.backing.node-name=node2,backing.backing.backing.backing.backing.node-name=node3,backing.backing.backing.backing.node-name=node4,backing.backing.backing.node-name=node5,backing.backing.node-name=node6,backing.node-name=node7,node-name=node8"
> +EE.............
> +======================================================================
> +ERROR: test_stream_parallel (__main__.TestParallelOps)
> +----------------------------------------------------------------------
> +Traceback (most recent call last):
> + File "tests/qemu-iotests/030", line 260, in test_stream_parallel
> + for event in self.vm.get_qmp_events(wait=True):
> + File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 597, in get_qmp_events
> + events = self._qmp.get_events(wait=wait)
> + File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 338, in get_events
> + self.__get_events(wait)
> + File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 207, in __get_events
> + raise QMPConnectError("Error while reading from socket")
> +qemu.qmp.QMPConnectError: Error while reading from socket
> +
> +======================================================================
> +ERROR: test_stream_parallel (__main__.TestParallelOps)
> +----------------------------------------------------------------------
> +Traceback (most recent call last):
> + File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 489, in _do_shutdown
> + self._soft_shutdown(timeout, has_quit)
> + File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 469, in _soft_shutdown
> + self._qmp.cmd('quit')
> + File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 288, in cmd
> + return self.cmd_obj(qmp_cmd)
> + File "tests/qemu-iotests/../../python/qemu/qmp/__init__.py", line 266, in cmd_obj
> + self.__sock.sendall(json.dumps(qmp_cmd).encode('utf-8'))
> +BrokenPipeError: [Errno 32] Broken pipe
> +
> +The above exception was the direct cause of the following exception:
> +
> +Traceback (most recent call last):
> + File "tests/qemu-iotests/030", line 227, in tearDown
> + self.vm.shutdown()
> + File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 519, in shutdown
> + self._do_shutdown(timeout, has_quit)
> + File "tests/qemu-iotests/../../python/qemu/machine/machine.py", line 492, in _do_shutdown
> + raise AbnormalShutdown("Could not perform graceful shutdown") \
> +qemu.machine.machine.AbnormalShutdown: Could not perform graceful shutdown
> +
> ----------------------------------------------------------------------
> Ran 27 tests
> -OK
> +FAILED (errors=2)
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-06-10 12:52 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-10 7:54 qemu io test: spurious failures of 030 Christian Borntraeger
2021-06-10 12:50 ` Vladimir Sementsov-Ogievskiy
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).