From: shaozhengchao <shaozhengchao@huawei.com>
To: Xin Long <lucien.xin@gmail.com>,
Marcelo Ricardo Leitner <marcelo.leitner@gmail.com>
Cc: <linux-sctp@vger.kernel.org>, <netdev@vger.kernel.org>,
<vyasevich@gmail.com>, <nhorman@tuxdriver.com>,
<davem@davemloft.net>, <edumazet@google.com>, <kuba@kernel.org>,
<pabeni@redhat.com>, <weiyongjun1@huawei.com>,
<yuehaibing@huawei.com>,
Tetsuo Handa <penguin-kernel@i-love.sakura.ne.jp>
Subject: Re: [PATCH net] sctp: fix memory leak in sctp_stream_outq_migrate()
Date: Thu, 24 Nov 2022 12:35:26 +0800 [thread overview]
Message-ID: <6a9555ac-92ba-cb25-4591-2af74d91264c@huawei.com> (raw)
In-Reply-To: <CADvbK_fyvw+dLN58V7kvZTn9tPT1UcL9kbguUjKh83urAD=Gxg@mail.gmail.com>
On 2022/11/24 11:04, Xin Long wrote:
> On Wed, Nov 23, 2022 at 2:01 PM Marcelo Ricardo Leitner
> <marcelo.leitner@gmail.com> wrote:
>>
>> On Wed, Nov 23, 2022 at 01:48:01PM -0500, Xin Long wrote:
>>> On Wed, Nov 23, 2022 at 1:30 PM Xin Long <lucien.xin@gmail.com> wrote:
>>>>
>>>> (
>>>>
>>>> On Wed, Nov 23, 2022 at 1:10 PM Marcelo Ricardo Leitner
>>>> <marcelo.leitner@gmail.com> wrote:
>>>>>
>>>>> On Wed, Nov 23, 2022 at 12:20:44PM -0500, Xin Long wrote:
>>>>>> On Tue, Nov 22, 2022 at 6:35 PM Marcelo Ricardo Leitner
>>>>>> <marcelo.leitner@gmail.com> wrote:
>>>>>>>
>>>>>>> On Fri, Nov 18, 2022 at 10:15:50PM -0500, Xin Long wrote:
>>>>>>>> On Fri, Nov 18, 2022 at 3:48 AM Zhengchao Shao <shaozhengchao@huawei.com> wrote:
>>>>>>>>>
>>>>>>>>> When sctp_stream_outq_migrate() is called to release stream out resources,
>>>>>>>>> the memory pointed to by prio_head in stream out is not released.
>>>>>>>>>
>>>>>>>>> The memory leak information is as follows:
>>>>>>>>> unreferenced object 0xffff88801fe79f80 (size 64):
>>>>>>>>> comm "sctp_repo", pid 7957, jiffies 4294951704 (age 36.480s)
>>>>>>>>> hex dump (first 32 bytes):
>>>>>>>>> 80 9f e7 1f 80 88 ff ff 80 9f e7 1f 80 88 ff ff ................
>>>>>>>>> 90 9f e7 1f 80 88 ff ff 90 9f e7 1f 80 88 ff ff ................
>>>>>>>>> backtrace:
>>>>>>>>> [<ffffffff81b215c6>] kmalloc_trace+0x26/0x60
>>>>>>>>> [<ffffffff88ae517c>] sctp_sched_prio_set+0x4cc/0x770
>>>>>>>>> [<ffffffff88ad64f2>] sctp_stream_init_ext+0xd2/0x1b0
>>>>>>>>> [<ffffffff88aa2604>] sctp_sendmsg_to_asoc+0x1614/0x1a30
>>>>>>>>> [<ffffffff88ab7ff1>] sctp_sendmsg+0xda1/0x1ef0
>>>>>>>>> [<ffffffff87f765ed>] inet_sendmsg+0x9d/0xe0
>>>>>>>>> [<ffffffff8754b5b3>] sock_sendmsg+0xd3/0x120
>>>>>>>>> [<ffffffff8755446a>] __sys_sendto+0x23a/0x340
>>>>>>>>> [<ffffffff87554651>] __x64_sys_sendto+0xe1/0x1b0
>>>>>>>>> [<ffffffff89978b49>] do_syscall_64+0x39/0xb0
>>>>>>>>> [<ffffffff89a0008b>] entry_SYSCALL_64_after_hwframe+0x63/0xcd
>>>>>>>>>
>>>>>>>>> Fixes: 637784ade221 ("sctp: introduce priority based stream scheduler")
>>>>>>>>> Reported-by: syzbot+29c402e56c4760763cc0@syzkaller.appspotmail.com
>>>>>>>>> Signed-off-by: Zhengchao Shao <shaozhengchao@huawei.com>
>>>>>>>>> ---
>>>>>>>>> net/sctp/stream.c | 6 ++++++
>>>>>>>>> 1 file changed, 6 insertions(+)
>>>>>>>>>
>>>>>>>>> diff --git a/net/sctp/stream.c b/net/sctp/stream.c
>>>>>>>>> index ef9fceadef8d..a17dc368876f 100644
>>>>>>>>> --- a/net/sctp/stream.c
>>>>>>>>> +++ b/net/sctp/stream.c
>>>>>>>>> @@ -70,6 +70,9 @@ static void sctp_stream_outq_migrate(struct sctp_stream *stream,
>>>>>>>>> * sctp_stream_update will swap ->out pointers.
>>>>>>>>> */
>>>>>>>>> for (i = 0; i < outcnt; i++) {
>>>>>>>>> + if (SCTP_SO(new, i)->ext)
>>>>>>>>> + kfree(SCTP_SO(new, i)->ext->prio_head);
>>>>>>>>> +
>>>>>>>>> kfree(SCTP_SO(new, i)->ext);
>>>>>>>>> SCTP_SO(new, i)->ext = SCTP_SO(stream, i)->ext;
>>>>>>>>> SCTP_SO(stream, i)->ext = NULL;
>>>>>>>>> @@ -77,6 +80,9 @@ static void sctp_stream_outq_migrate(struct sctp_stream *stream,
>>>>>>>>> }
>>>>>>>>>
>>>>>>>>> for (i = outcnt; i < stream->outcnt; i++) {
>>>>>>>>> + if (SCTP_SO(stream, i)->ext)
>>>>>>>>> + kfree(SCTP_SO(stream, i)->ext->prio_head);
>>>>>>>>> +
>>>>>>>>> kfree(SCTP_SO(stream, i)->ext);
>>>>>>>>> SCTP_SO(stream, i)->ext = NULL;
>>>>>>>>> }
>>>>>>>>> --
>>>>>>>>> 2.17.1
>>>>>>>>>
>>>>>>>> This is not a proper fix:
>>>>>>>> 1. you shouldn't access "prio_head" outside stream_sched_prio.c.
>>>>>>>> 2. the prio_head you freed might be used by other out streams, freeing
>>>>>>>> it unconditionally would cause either a double free or use after free.
>>>>>>>>
>>>>>>>> I'm afraid we have to add a ".free_sid" in sctp_sched_ops, and
>>>>>>>> implement it for sctp_sched_prio, like:
>>>>>>>>
>>>>>>>> +static void sctp_sched_prio_free_sid(struct sctp_stream *stream, __u16 sid)
>>>>>>>> +{
>>>>>>>> + struct sctp_stream_priorities *prio = SCTP_SO(stream,
>>>>>>>> sid)->ext->prio_head;
>>>>>>>> + int i;
>>>>>>>> +
>>>>>>>> + if (!prio)
>>>>>>>> + return;
>>>>>>>> +
>>>>>>>> + SCTP_SO(stream, sid)->ext->prio_head = NULL;
>>>>>>>> + for (i = 0; i < stream->outcnt; i++) {
>>>>>>>
>>>>>>> Instead of checking all streams, the for() can/should be replaced by
>>>>>>> (from sctp_sched_prio_free):
>>>>>>> if (!list_empty(&prio->prio_sched))
>>>>>>> return;
>>>>>> sctp_stream_outq_migrate() is called after unsched_all() for "stream",
>>>>>> list_empty(prio_sched) is expected to be true.
>>>>>
>>>>> Good point. Am I missing something or the 'prio_head == prio' below
>>>>> would always be false then as well?
>>> sorry, forgot to reply to this one :D
>>
>> :D
>>
>>>
>>> after .unsched_all, multiple outstreams may have the same prio_head,
>>> which are not on any list (like stream->prio_list).
>>>
>>> so when freeing one outstream ext, it will need to go over all outstreams' exts
>>> and check if this outstream ext's prio is equal to that of any other outstreams.
>>
>> Understood. The check in sctp_sched_prio_free() is actually checking
>> if the prio_head is not yet scheduled for freeing instead, right.
>> Thanks. Hmm. This for() can be quite expensive then. :-(
>>
>>>
>>>>>
>>>>> Anyhow, as this is moving to something that can potentially be called
>>>>> from other places afterwards, keeping the check doesn't hurt.
>>>>>
>>>>>>
>>>>>> Note that kfree(SCTP_SO(new, i)->ext) shouldn't have the reported
>>>>>> problem, as at that moment, the "new" stream hasn't been set
>>>>>> stream_sched yet. It means there's only one place that needs to
>>>>>> call free_sid in sctp_stream_outq_migrate().
>>>>>> (Maybe Zhengchao can help us confirm this?)
>>>>>
>>>>> That's the case in Tetsuo's patch (earlier today) as well. Yet, if we
>>>>> have an official way to free a stream, if it's not error handling
>>>>> during initialization, it should use it.
>>>> right.
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>> + if (SCTP_SO(stream, i)->ext &&
>>>>>>>> + SCTP_SO(stream, i)->ext->prio_head == prio)
>>>>>>>> + return;
>>>>>>>> + }
>>>>>>>> + kfree(prio);
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> static void sctp_sched_prio_free(struct sctp_stream *stream)
>>>>>>>> {
>>>>>>>> struct sctp_stream_priorities *prio, *n;
>>>>>>>> @@ -323,6 +340,7 @@ static struct sctp_sched_ops sctp_sched_prio = {
>>>>>>>> .get = sctp_sched_prio_get,
>>>>>>>> .init = sctp_sched_prio_init,
>>>>>>>> .init_sid = sctp_sched_prio_init_sid,
>>>>>>>> + .free_sid = sctp_sched_prio_free_sid,
>>>>>>>> .free = sctp_sched_prio_free,
>>>>>>>> .enqueue = sctp_sched_prio_enqueue,
>>>>>>>> .dequeue = sctp_sched_prio_dequeue,
>>>>>>>>
>>>>>>>> then call it in sctp_stream_outq_migrate(), like:
>>>>>>>>
>>>>>>>> +static void sctp_stream_free_ext(struct sctp_stream *stream, __u16 sid)
>>>>>>>> +{
>>>>>>>> + struct sctp_sched_ops *sched = sctp_sched_ops_from_stream(stream);
>>>>>>>> +
>>>>>>>> + sched->free_sid(stream, sid);
>>>>>>>> + kfree(SCTP_SO(stream, sid)->ext);
>>>>>>>> + SCTP_SO(stream, sid)->ext = NULL;
>>>>>>>> +}
>>>>>>>> +
>>>>>>>> /* Migrates chunks from stream queues to new stream queues if needed,
>>>>>>>> * but not across associations. Also, removes those chunks to streams
>>>>>>>> * higher than the new max.
>>>>>>>> @@ -70,16 +79,14 @@ static void sctp_stream_outq_migrate(struct
>>>>>>>> sctp_stream *stream,
>>>>>>>> * sctp_stream_update will swap ->out pointers.
>>>>>>>> */
>>>>>>>> for (i = 0; i < outcnt; i++) {
>>>>>>>> - kfree(SCTP_SO(new, i)->ext);
>>>>>>>> + sctp_stream_free_ext(new, i);
>>>>>>>> SCTP_SO(new, i)->ext = SCTP_SO(stream, i)->ext;
>>>>>>>> SCTP_SO(stream, i)->ext = NULL;
>>>>>>>> }
>>>>>>>> }
>>>>>>>>
>>>>>>>> - for (i = outcnt; i < stream->outcnt; i++) {
>>>>>>>> - kfree(SCTP_SO(stream, i)->ext);
>>>>>>>> - SCTP_SO(stream, i)->ext = NULL;
>>>>>>>> - }
>>>>>>>> + for (i = outcnt; i < stream->outcnt; i++)
>>>>>>>> + sctp_stream_free_ext(new, i);
>>>>>>>> }
>>>>>>>>
>>>>>>>> Marcelo, do you see a better solution?
>>>>>>>
>>>>>>> No. Your suggestion is the best I could think of too.
>>>>>>>
>>>>>>> Another approach would be to expose sched->free and do all the freeing
>>>>>>> at once, like sctp_stream_free() does. But the above is looks cleaner
>>>>>>> and makes it evident that freeing 'ext' is not trivial.
>>>>>>>
>>>>>>> With the proposal above, sctp_sched_prio_free() becomes an
>>>>>>> optimization, if we can call it that. With the for/if replacement
>>>>>>> above, not even that, and should be removed. Including sctp_sched_ops
>>>>>>> 'free' pointer.
>>>>>> Or we extract the common code to another function, like
>>>>>> sctp_sched_prio_free_head(stream, prio), and pass prio as
>>>>>> NULL in sctp_sched_prio_free() for freeing all.
>>>>>>
>>>>>>>
>>>>>>> sctp_stream_free() then should be updated to use the new
>>>>>>> sctp_stream_free_ext() instead, instead of mangling it directly.
>>>>>> I thought about this, but there is ".free", which is more efficient
>>>>>> to free all prio than calling ".free_sid" outcnt times.
>>>>>
>>>>> How much more efficient, just by avoiding retpoline stuff on the
>>>>> indirect functional call or something else?
>>>>
>>>> in sctp_stream_free():
>>>> .free() will be called one time to free all prios
>>>> while .free_sid will be called in a loop to free all prios:
>>>> for (i = 0; i < stream->outcnt; i++)
>>>> .free_sid(stream, i);
>>>>
>>>> inside either() .free or . free_sid() there is another loop:
>>>> for (i = 0; i < stream->outcnt; i++)
>>>> ...
>>>>
>>>> That's why I said using .free() in sctp_stream_free() will be more efficient.
>>>>
>>>>>
>>>>>>
>>>>>> I may move free_sid() out of sctp_stream_free_ext(), then in
>>>>>> sctp_stream_free() we can call sctp_stream_free_ext() without
>>>>>> calling free_sid(), or just remove sctp_stream_free_ext().
>>>>>
>>>>> It's easier to maintain it if we have symmetric paths for initializing
>>>>> and for freeing it and less special cases. We already have
>>>>> sctp_stream_init_ext(), so having sctp_stream_free_ext() is not off.
>>>> didn't notice init_sid in sctp_stream_init_ext(), it makes sense to
>>>> have free_sid in sctp_stream_free_ext().
>>>>
>>>> Thanks.
>>>>
>>>>>
>>>>> I'm happy to review any patch that also updates sctp_stream_free(),
>>>>> one way or another.
>>>>>
> Hi, Zhengchao
>
> Would you please post v2 with the proposal above?
> (also add syzkaller-bugs@googlegroups.com into Cc list as Tetsuo
> suggested in another thread)
>
> Thanks.
OK， I will send V2 as soon as I have tested patch.
Zhengchao Shao
prev parent reply other threads:[~2022-11-24 4:35 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-18 8:50 [PATCH net] sctp: fix memory leak in sctp_stream_outq_migrate() Zhengchao Shao
2022-11-19 3:15 ` Xin Long
2022-11-22 23:35 ` Marcelo Ricardo Leitner
2022-11-23 17:20 ` Xin Long
2022-11-23 18:10 ` Marcelo Ricardo Leitner
2022-11-23 18:30 ` Xin Long
2022-11-23 18:48 ` Xin Long
2022-11-23 19:01 ` Marcelo Ricardo Leitner
2022-11-24 3:04 ` Xin Long
2022-11-24 4:35 ` shaozhengchao [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=6a9555ac-92ba-cb25-4591-2af74d91264c@huawei.com \
--to=shaozhengchao@huawei.com \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=kuba@kernel.org \
--cc=linux-sctp@vger.kernel.org \
--cc=lucien.xin@gmail.com \
--cc=marcelo.leitner@gmail.com \
--cc=netdev@vger.kernel.org \
--cc=nhorman@tuxdriver.com \
--cc=pabeni@redhat.com \
--cc=penguin-kernel@i-love.sakura.ne.jp \
--cc=vyasevich@gmail.com \
--cc=weiyongjun1@huawei.com \
--cc=yuehaibing@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).