From: Pavel Pisa <pisa@fel.cvut.cz>
To: Oliver Hartkopp <socketcan@hartkopp.net>
Cc: linux-can@vger.kernel.org, linux-rt-users@vger.kernel.org,
"Ondrej Ille" <ondrej.ille@gmail.com>,
"Matěj Vasilevski" <matej.vasilevski@gmail.com>,
"Pavel Hronek" <hronepa1@fel.cvut.cz>,
"Jiří Novák" <jnovak@fel.cvut.cz>,
"Carsten Emde" <c.emde@osadl.org>,
"Jan Altenberg" <Jan.Altenberg@osadl.org>
Subject: Re: Outstanding latency increase in kernel CAN gateway caught by CANlatester daily builds at 2023-10-02
Date: Wed, 18 Oct 2023 11:50:56 +0200 [thread overview]
Message-ID: <202310181150.56364.pisa@fel.cvut.cz> (raw)
In-Reply-To: <383f33ac-3254-4277-a9a9-8f7104851d49@hartkopp.net>
Hello Oliver,
On Saturday 14 of October 2023 12:57:53 Oliver Hartkopp wrote:
> Hello Pavel,
>
> is there any news on this latency issue?
>
> I've not seen any can-gw related changes between 6.2 and 6.6.
>
> The only change for linux/net/can/gw.c is this patch:
>
> https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?
>id=2a30b2bd01c23a7eeace3a3f82c2817227099805
I am keeping eye on the results daily and it seems that run
run-231002-045216-hist+6.6.0-rc3-rt5-ge31516c1e553+flood-kern-prio-fd-load
https://canbus.pages.fel.cvut.cz/can-latester/inspect.html?kernel=rt&load=1&flood=1&fd=1&prio=1&kern=1
is really outlayer. At the end it is single one, I have
interpretted the graph incorrectly because so much outstanding
value at the end looked as flat increase covering two consecutive
runs.
So the kernel GW average and maximas follow previous trend
after this single peak. So the peak could be related to some
transitional state in RT development causing some problem
with priorities etc., or can be result of some other problems
in the whole setup. I am analyzing some problems with
lost messages in some cases of RT runs which seems
to be related more to some problem in testing system,
setup before run, FPGA reload etc. which cause bus error
or something similar with initial suspicion on monitoring
side problem. But I do not have conclusion yet.
Published runs are complete with no message lost
and statistic/trends seems to be without significant
change from start of the measurement in May.
The change/increase of trends before May has well understood
reason, we have updated stress testing, include more sources
and tuned priorities for user GW etc...
In alonger term perspective, initial setup testing data from
April should/will be removed/masked from public data to not
provide false assumptions. We probably start new series
when 2024 year starts. We will see how data capacity
and viewing will work work and slower as data set is extended.
If I notice some significant change in more consecutive runs,
I try to check it and send information.
In the fact, we have caught one real problem in RT
already.
Best wishes,
Pavel
--
Pavel Pisa
phone: +420 603531357
e-mail: pisa@cmp.felk.cvut.cz
Department of Control Engineering FEE CVUT
Karlovo namesti 13, 121 35, Prague 2
university: http://control.fel.cvut.cz/
personal: http://cmp.felk.cvut.cz/~pisa
social: https://social.kernel.org/ppisa
projects: https://www.openhub.net/accounts/ppisa
CAN related:http://canbus.pages.fel.cvut.cz/
RISC-V education: https://comparch.edu.cvut.cz/
Open Technologies Research Education and Exchange Services
https://gitlab.fel.cvut.cz/otrees/org/-/wikis/home
prev parent reply other threads:[~2023-10-18 9:51 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-10-02 8:40 Outstanding latency increase in kernel CAN gateway caught by CANlatester daily builds at 2023-10-02 Pavel Pisa
2023-10-14 10:57 ` Oliver Hartkopp
2023-10-18 9:50 ` Pavel Pisa [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=202310181150.56364.pisa@fel.cvut.cz \
--to=pisa@fel.cvut.cz \
--cc=Jan.Altenberg@osadl.org \
--cc=c.emde@osadl.org \
--cc=hronepa1@fel.cvut.cz \
--cc=jnovak@fel.cvut.cz \
--cc=linux-can@vger.kernel.org \
--cc=linux-rt-users@vger.kernel.org \
--cc=matej.vasilevski@gmail.com \
--cc=ondrej.ille@gmail.com \
--cc=socketcan@hartkopp.net \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).