From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id D78D01F910 for ; Wed, 30 Nov 2022 06:49:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1669790970; bh=LO0zBGneE0u81khk9W0Dfds2Ef7HRWf3rgvZp38ChmI=; h=From:To:Subject:Date:In-Reply-To:References:From; b=zd3VbT/8ZMjvue2cKhS9nvkJbHIGh5enuBg3rf90BYAyFxL5z5v2/HPVp7XB+KSkV YZFjeq+L1skDrc+QWkykwdY2qgdZ90wnCLbimHjf2BpEGTBvjGdZtvgaxFAmAOAghw zvqUg6oCO29tOfybn2cFM9j41t43JkICY17EwSQk= From: Eric Wong To: mwrap-perl@80x24.org Subject: [PATCH 1/2] use file-backed mmap on Linux Date: Wed, 30 Nov 2022 06:49:29 +0000 Message-Id: <20221130064930.5239-2-e@80x24.org> In-Reply-To: <20221130064930.5239-1-e@80x24.org> References: <20221130064930.5239-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: Due to the high memory usage of mwrap, it may be useful to force allocations onto file-backed maps. This means more physical memory for the rest of the system. As opposed to adding swap indiscriminately, this provides the ability to give the equivalent of swap space only to the processes using mwrap. O_TMPFILE in Linux makes supporting this a breeze, since we can't expect to use tmpfile(3) before malloc is initialized. --- mymalloc.h | 44 ++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 44 insertions(+) diff --git a/mymalloc.h b/mymalloc.h index 4904b74..f5a171e 100644 --- a/mymalloc.h +++ b/mymalloc.h @@ -24,6 +24,46 @@ #include #include #include +#include +#include +#include +#include + +#if !defined(MWRAP_FILE_BACKED) && defined(__linux__) && defined(O_TMPFILE) +# define MWRAP_FILE_BACKED 1 +#else +# define MWRAP_FILE_BACKED 0 +#endif +#if MWRAP_FILE_BACKED +# include +static void *my_mmap(size_t size) +{ + int flags = MAP_PRIVATE; + const char *tmpdir = getenv("TMPDIR"); + int fd; + void *ret; + + if (!tmpdir) + tmpdir = "/tmp"; + + fd = open(tmpdir, O_TMPFILE|O_RDWR|S_IRUSR); + if (fd < 0) { + flags |= MAP_ANONYMOUS; + } else { + ssize_t w = pwrite(fd, "", 1, size - 1); /* sparse file */ + + if (w < 0) + flags |= MAP_ANONYMOUS; + } + ret = mmap(NULL, size, PROT_READ|PROT_WRITE, flags, fd, 0); + if (fd >= 0) { + int err = errno; + close(fd); + errno = err; + } + return ret; +} +#endif /* MWRAP_FILE_BACKED */ /* knobs for dlmalloc */ #define FOOTERS 1 /* required for remote_free_* stuff */ @@ -32,6 +72,10 @@ #define DLMALLOC_EXPORT static inline /* #define NO_MALLOC_STATS 1 */ #define USE_LOCKS 0 /* we do our own global_mtx + ms_tsd */ +#if MWRAP_FILE_BACKED +# define MMAP(size) my_mmap(size) +# define HAVE_MREMAP 0 +#endif #include "dlmalloc_c.h" #undef ABORT /* conflicts with Perl */