From: Eric Wong <e@80x24.org>
To: mwrap-perl@80x24.org
Subject: [PATCH 3/3] malloc: revert file-backed mmap, sbrk for main arena
Date: Wed, 3 Apr 2024 21:42:22 +0000 [thread overview]
Message-ID: <20240403214222.3258695-4-e@80x24.org> (raw)
In-Reply-To: <20240403214222.3258695-1-e@80x24.org>
With the effectiveness of anti-fragmentation size classes and
subsequent memory reduction, the non-growable file-backed mmap
to be the backing store makes little sense.
---
dlmalloc_c.h | 13 ++++++------
mymalloc.h | 56 ++--------------------------------------------------
2 files changed, 9 insertions(+), 60 deletions(-)
diff --git a/dlmalloc_c.h b/dlmalloc_c.h
index 348c194..5aa9e94 100644
--- a/dlmalloc_c.h
+++ b/dlmalloc_c.h
@@ -4319,11 +4319,9 @@ static int sys_trim(mstate m, size_t pad) {
!has_segment_link(m, sp)) { /* can't shrink if pinned */
size_t newsize = sp->size - extra;
(void)newsize; /* placate people compiling -Wunused-variable */
- /*
- * dlmalloc preferred mremap, here, but mwrap uses a file-backed
- * mmap for small allocations on Linux and we close the FD ASAP.
- */
- if (CALL_MUNMAP(sp->base + newsize, extra) == 0) {
+ /* Prefer mremap, fall back to munmap */
+ if ((CALL_MREMAP(sp->base, sp->size, newsize, 0) != MFAIL) ||
+ (CALL_MUNMAP(sp->base + newsize, extra) == 0)) {
released = extra;
}
}
@@ -5422,7 +5420,10 @@ static mstate init_user_mstate(char* tbase, size_t tsize) {
m->magic = mparams.magic;
m->release_checks = MAX_RELEASE_CHECK_RATE;
m->mflags = mparams.default_mflags;
- disable_contiguous(m);
+ static int once;
+# define MP_FIRST_TIME (uatomic_cmpxchg(&once, 0, 1) == 0)
+ if (!MP_FIRST_TIME)
+ disable_contiguous(m);
init_bins(m);
mn = next_chunk(mem2chunk(m));
init_top(m, mn, (size_t)((tbase + tsize) - (char*)mn) - TOP_FOOT_SIZE);
diff --git a/mymalloc.h b/mymalloc.h
index 44c5dd3..4dd2ee6 100644
--- a/mymalloc.h
+++ b/mymalloc.h
@@ -40,68 +40,16 @@
#include <urcu/rculist.h>
#include <urcu/wfcqueue.h>
#include <urcu-bp.h>
-#include <sys/types.h>
-#include <sys/stat.h>
-#include <fcntl.h>
-#include <errno.h>
-
-/* this is fine on most x86-64, especially with file-backed mmap(2) */
-#define DEFAULT_GRANULARITY (64U * 1024U * 1024U)
-
-#if !defined(MWRAP_FILE_BACKED) && defined(__linux__) && defined(O_TMPFILE)
-# define MWRAP_FILE_BACKED 1
-#else
-# define MWRAP_FILE_BACKED 0
-#endif
-#if MWRAP_FILE_BACKED
-# include <sys/mman.h>
-static void *my_mmap(size_t size)
-{
- int flags = MAP_PRIVATE;
- const char *tmpdir = getenv("TMPDIR");
- int fd;
- void *ret;
-
- if (!tmpdir)
- tmpdir = "/tmp";
-
- fd = open(tmpdir, O_TMPFILE|O_RDWR|S_IRUSR|S_IWUSR, 0600);
- if (fd < 0) {
- flags |= MAP_ANONYMOUS;
- } else {
- int t = ftruncate(fd, size); /* sparse file */
-
- if (t < 0) {
- flags |= MAP_ANONYMOUS;
- close(fd);
- fd = -1;
- }
- }
- ret = mmap(NULL, size, PROT_READ|PROT_WRITE, flags, fd, 0);
- if (fd >= 0) {
- int err = errno;
- close(fd);
- errno = err;
- }
- return ret;
-}
-#endif /* MWRAP_FILE_BACKED */
/* knobs for dlmalloc */
+#define HAVE_MORECORE 1
+#define DEFAULT_GRANULARITY (2U * 1024U * 1024U)
#define FOOTERS 1 /* required for remote_free_* stuff */
#define USE_DL_PREFIX
#define ONLY_MSPACES 1 /* aka per-thread "arenas" */
#define DLMALLOC_EXPORT static inline
/* #define NO_MALLOC_STATS 1 */
#define USE_LOCKS 0 /* we do our own global_mtx + ms_tsd */
-#if MWRAP_FILE_BACKED
-# define MMAP(size) my_mmap(size)
-/*
- * we use anonymous mmap (via DIRECT_MMAP) and mremap since it benefits
- * large allocations which require resizing (open-addressed hash tables,
- * large string buffers, etc)
- */
-#endif
#include "dlmalloc_c.h"
#undef ABORT /* conflicts with Perl */
#undef NOINLINE /* conflicts with Ruby, defined by dlmalloc_c.h */
prev parent reply other threads:[~2024-04-03 21:42 UTC|newest]
Thread overview: 4+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-03 21:42 [PATCH 0/3] introduce size classes to reduce fragmentation Eric Wong
2024-04-03 21:42 ` [PATCH 1/3] dlmalloc: use jemalloc-inspired size classes Eric Wong
2024-04-03 21:42 ` [PATCH 2/3] implement dynamic mmap threshold Eric Wong
2024-04-03 21:42 ` Eric Wong [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240403214222.3258695-4-e@80x24.org \
--to=e@80x24.org \
--cc=mwrap-perl@80x24.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
Code repositories for project(s) associated with this public inbox
https://80x24.org/mwrap-perl.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).