From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 762F01F626 for ; Wed, 22 Feb 2023 15:19:12 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1677079152; bh=pKvHKGWUTQrAD33rAbAo9+Tj4OlKNC4WlWFCZu/60NE=; h=From:To:Subject:Date:From; b=OQzzU34t5NlMViNxpEoowY23gja2vsegv1xAAl1kcDAFwZWDMWquo+n6l9Fa0QR7N IFJHuTg32PeTzYg8bzK8IsrSmInkgujIyHa34AfUD60z251Bl2tBjrDPZAH9i7L0gM cnwSvgPKkNPuAq/lvN1ajN1MQL48QauA0MCIHBKg= From: Eric Wong To: mwrap-perl@80x24.org Subject: [PATCH] allow mremap and MAP_ANONYMOUS for large allocations Date: Wed, 22 Feb 2023 15:19:12 +0000 Message-Id: <20230222151912.749652-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Our use of file-backed mmap on Linux benefits small allocations since our overhead is most noticeable there. Allocations requiring mmap on a normal allocator are large enough that the overhead from mwrap makes no difference. However, Linux mremap can provide a noticeable improvement when growing arrays quickly, such as those used for open address hash tables or giant strings in HTML rendering. This reverts 2d31f30386fa848b93a72bc2e4329e966026ab45 (mymalloc: use file-backed DIRECT_MMAP, too, 2023-02-01) --- dlmalloc_c.h | 8 +++++--- mymalloc.h | 7 +++++-- 2 files changed, 10 insertions(+), 5 deletions(-) diff --git a/dlmalloc_c.h b/dlmalloc_c.h index 38a0846..e842a53 100644 --- a/dlmalloc_c.h +++ b/dlmalloc_c.h @@ -4319,9 +4319,11 @@ static int sys_trim(mstate m, size_t pad) { !has_segment_link(m, sp)) { /* can't shrink if pinned */ size_t newsize = sp->size - extra; (void)newsize; /* placate people compiling -Wunused-variable */ - /* Prefer mremap, fall back to munmap */ - if ((CALL_MREMAP(sp->base, sp->size, newsize, 0) != MFAIL) || - (CALL_MUNMAP(sp->base + newsize, extra) == 0)) { + /* + * dlmalloc preferred mremap, here, but mwrap uses a file-backed + * mmap for small allocations on Linux and we close the FD ASAP. + */ + if (CALL_MUNMAP(sp->base + newsize, extra) == 0) { released = extra; } } diff --git a/mymalloc.h b/mymalloc.h index 773d3ce..44c5dd3 100644 --- a/mymalloc.h +++ b/mymalloc.h @@ -96,8 +96,11 @@ static void *my_mmap(size_t size) #define USE_LOCKS 0 /* we do our own global_mtx + ms_tsd */ #if MWRAP_FILE_BACKED # define MMAP(size) my_mmap(size) -# define DIRECT_MMAP(size) my_mmap(size) -# define HAVE_MREMAP 0 +/* + * we use anonymous mmap (via DIRECT_MMAP) and mremap since it benefits + * large allocations which require resizing (open-addressed hash tables, + * large string buffers, etc) + */ #endif #include "dlmalloc_c.h" #undef ABORT /* conflicts with Perl */