From: Russell Johnson <russell.johnson@kratosdefense.com>
To: Philippe Gerum <rpm@xenomai.org>
Cc: "xenomai@lists.linux.dev" <xenomai@lists.linux.dev>,
Dave Rolenc <Dave.Rolenc@kratosdefense.com>
Subject: RE: [External] - Re: CPU Stuck Error When Using Xbufs
Date: Wed, 23 Aug 2023 21:42:51 +0000 [thread overview]
Message-ID: <PH1P110MB10506D3909C7F85BC5523BFAE21CA@PH1P110MB1050.NAMP110.PROD.OUTLOOK.COM> (raw)
In-Reply-To: <87bkeyp3hs.fsf@xenomai.org>
[-- Attachment #1: Type: text/plain, Size: 2993 bytes --]
Hi Philippe,
It appears that this has solved our issue. Thanks!
Russell
-----Original Message-----
From: Philippe Gerum <rpm@xenomai.org>
Sent: Wednesday, August 23, 2023 2:26 AM
To: Russell Johnson <russell.johnson@kratosdefense.com>
Cc: xenomai@lists.linux.dev; Dave Rolenc <Dave.Rolenc@kratosdefense.com>
Subject: [External] - Re: CPU Stuck Error When Using Xbufs
CAUTION: This email originated from outside of the organization. Do not
click links or open attachments unless you recognize the sender and know the
content is safe.
Philippe Gerum <rpm@xenomai.org> writes:
> Russell Johnson <russell.johnson@kratosdefense.com> writes:
>
>> [[S/MIME Signed Part:Undecided]]
>> Hello,
>>
>> I am running into a CPU Stuck issue when I added another xbuf to our
>> application where we have data being transported from EVL to Linux
>> constantly.
>>
>> I get a message on the console that mentions there is a CPU that is
>> stuck due to spinlock recursion:
>> "
>> Message from syslogd@Modem1 at Aug 18 23:02:25 ...
>> kernel:BUG: spinlock cpu recursion on CPU#5, SINK_0_DATA/4437 Message
>> from syslogd@Modem1 at Aug 18 23:02:25 ...
>> kernel: lock: 0xffff8f27d329f610, .magic: dead4ead, .owner:
>> SINK_0_XBUF_L/4402, .owner_cpu: 5
>> "
>>
>> I collected some data using sysrq and got the following stacks for
>> both of the threads in question:
>> "
>> task:SINK_0_DATA state:S stack: 0 pid:15095 ppid: 14415
>> flags:0x00000000
>> Call Trace:
>> <TASK>
>> ? wait_events.isra.15+0xae/0x3e0
>> ? evl_lock_mutex_timeout+0x45/0x540
>> ? evl_flush_wait+0x50/0x50
>> ? monitor_oob_ioctl+0x487/0xbd0
>> ? poll_oob_ioctl+0xc7/0x440
>> ? EVL_ioctl+0x46/0xa0
>> ? do_oob_syscall+0x251/0x260
>> ? handle_oob_syscall+0xa4/0xf0
>> ? pipeline_syscall+0x65/0x130
>> ? syscall_enter_from_user_mode+0x4d/0x120
>> ? do_syscall_64+0x1d/0xa0
>> ? entry_SYSCALL_64_after_hwframe+0x67/0xd1
>> </TASK>
>>
>> task:SINK_0_XBUF_L state:R running task stack: 0 pid:15059
ppid:
>> 14333 flags:0x00000008
>> Call Trace:
>> <TASK>
>> dovetail_context_switch+0xaf/0x200
>> __evl_schedule+0x266/0x560
>> run_oob_call+0x99/0x170
>> inbound_signal_output+0xab/0xb0
>> do_xbuf_read+0x1b3/0x240
>> xbuf_read+0x4d/0x70
>> ? do_irq_work+0x70/0x70
>> vfs_read+0x9b/0x1a0
>> ksys_read+0xc4/0xe0
>> __x64_sys_read+0x15/0x20
>> do_syscall_64+0x4c/0xa0
>> entry_SYSCALL_64_after_hwframe+0x67/0xd1
>> "
>>
>> NOTE: SINK_0_DATA is the thread in the EVL app that is pushing data
>> onto the xbuf. SINK_0_XBUF_L is the thread in the Linux app that is
>> pulling data off of the xbuf.
>>
>> Do you have any idea would could be going on?
>
> Yes, quite clearly now. I'll follow up on this in the coming days.
>
Please give a try to this patch:
https://urldefense.com/v3/__https://source.denx.de/Xenomai/xenomai4/linux-ev
l/-/commit/6b0dbab59eedb0b992a08be35197fccacad578d3__;!!KM6X6ZXWXVtZMQ!aQX1I
dWU2mC0fpoFsdz9yOB0Ywl-whRYJ1j3vJ3bM_HXU4sIAzNqORFiWcC1mbs6HCAjWbqDyJsbvPYyk
ShlZA$
Thanks,
--
Philippe.
[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 6739 bytes --]
prev parent reply other threads:[~2023-08-23 21:42 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <PH1P110MB1050E89C04D3AE6B62BB426FE21FA@PH1P110MB1050.NAMP110.PROD.OUTLOOK.COM>
2023-08-22 22:02 ` CPU Stuck Error When Using Xbufs Russell Johnson
2023-08-23 7:29 ` Philippe Gerum
2023-08-23 8:25 ` Philippe Gerum
2023-08-23 21:42 ` Russell Johnson [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=PH1P110MB10506D3909C7F85BC5523BFAE21CA@PH1P110MB1050.NAMP110.PROD.OUTLOOK.COM \
--to=russell.johnson@kratosdefense.com \
--cc=Dave.Rolenc@kratosdefense.com \
--cc=rpm@xenomai.org \
--cc=xenomai@lists.linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).