From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.2 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF shortcircuit=no autolearn=ham autolearn_force=no version=3.4.6 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id C055A1F44D for ; Thu, 18 Apr 2024 11:22:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=80x24.org; s=selector1; t=1713439336; bh=q5nd7Je3yzCBXEtjbAfyPSwoA+yHg5mE+ceRtsRLioI=; h=From:To:Subject:Date:From; b=q5e+iIbU+xpBUrhgjhCv7I2qZB/+XwRmtm/X1eX2m7WTYsaX3QfHrGTusk8ChzThS GMXg59Muvsm/IkyGAT/cZSaM5VonZ5jsbCa7gxNqhuenByZOzbzWzuRTPZbqYJLVGh 3oRYUzvZ4VVXG0LRC6ZbO7IRVn+tlPSiVjUbENiE= From: Eric Wong To: spew@80x24.org Subject: [PATCH] doc: strongly recommend MALLOC_MMAP_THRESHOLD_=131072 for glibc Date: Thu, 18 Apr 2024 11:22:16 +0000 Message-Id: <20240418112216.44725-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit List-Id: The 131072 byte lower bound was the old default before the sliding mmap window was introduced in modern glibc malloc. Setting MALLOC_MMAP_THRESHOLD_=131072 in your environment While the sliding mmap window can be faster by reducing syscalls and kernel overhead, was also prone to fragmentation problems due to the large variablity of email sizes. Individual allocations over 100K are rare in our codebase since there aren't often messages this large. --- Documentation/public-inbox-tuning.pod | 9 ++++----- examples/public-inbox-httpd@.service | 1 + examples/public-inbox-imapd@.service | 2 ++ examples/public-inbox-netd@.service | 11 ++++++----- examples/public-inbox-nntpd@.service | 2 ++ 5 files changed, 15 insertions(+), 10 deletions(-) diff --git a/Documentation/public-inbox-tuning.pod b/Documentation/public-inbox-tuning.pod index 7d0690b4..892ee0f2 100644 --- a/Documentation/public-inbox-tuning.pod +++ b/Documentation/public-inbox-tuning.pod @@ -165,11 +165,10 @@ capacity planning. Bursts of small object allocations late in process life contribute to fragmentation of the heap due to arenas (slabs) used internally by Perl. -jemalloc (tested as an LD_PRELOAD on GNU/Linux) reduces -overall fragmentation compared to glibc malloc in long-lived processes. -glibc malloc users may try setting C to a lower -value (e.g. 131072) but that may require increasing the -C sysctl. +glibc malloc users should use C to reduce +fragmentation from the sliding mmap window. jemalloc (tested as an +LD_PRELOAD on GNU/Linux) also reduces fragmentation compared to an +unconfigured glibc malloc in long-lived processes. =head2 Other OS tuning knobs diff --git a/examples/public-inbox-httpd@.service b/examples/public-inbox-httpd@.service index 11859198..ca68fc7e 100644 --- a/examples/public-inbox-httpd@.service +++ b/examples/public-inbox-httpd@.service @@ -19,6 +19,7 @@ After = public-inbox-httpd.socket Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ TZ=UTC \ +MALLOC_MMAP_THRESHOLD_=131072 \ PERL_INLINE_DIRECTORY=/tmp/.pub-inline LimitNOFILE = 30000 diff --git a/examples/public-inbox-imapd@.service b/examples/public-inbox-imapd@.service index 80104605..1aede65d 100644 --- a/examples/public-inbox-imapd@.service +++ b/examples/public-inbox-imapd@.service @@ -16,6 +16,8 @@ After = public-inbox-imapd.socket [Service] Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ +TZ=UTC \ +MALLOC_MMAP_THRESHOLD_=131072 \ PERL_INLINE_DIRECTORY=/tmp/.pub-inline LimitNOFILE = 30000 diff --git a/examples/public-inbox-netd@.service b/examples/public-inbox-netd@.service index 7437f086..51f58fbb 100644 --- a/examples/public-inbox-netd@.service +++ b/examples/public-inbox-netd@.service @@ -12,14 +12,15 @@ Wants = public-inbox-netd.socket After = public-inbox-netd.socket [Service] -# An LD_PRELOAD for libjemalloc can be added here. It is -# more resistant to fragmentation in long-lived daemons than glibc. -# If you're unable to use jemalloc, setting MALLOC_MMAP_THRESHOLD_ -# to a lower value (e.g. 131072) but that may also require increasing -# the sys.vm.max_map_count sysctl. + +# Setting MALLOC_MMAP_THRESHOLD_=131072 reduces fragmentation by +# disabling the sliding mmap window in glibc malloc. An LD_PRELOAD for +# libjemalloc may be added here, instead. jemalloc is more resistant to +# fragmentation in long-lived daemons than unconfigured glibc malloc. Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ TZ=UTC \ +MALLOC_MMAP_THRESHOLD_=131072 \ PERL_INLINE_DIRECTORY=/tmp/.netd-inline LimitNOFILE = 30000 diff --git a/examples/public-inbox-nntpd@.service b/examples/public-inbox-nntpd@.service index 24f9ca73..556cb76f 100644 --- a/examples/public-inbox-nntpd@.service +++ b/examples/public-inbox-nntpd@.service @@ -16,6 +16,8 @@ After = public-inbox-nntpd.socket [Service] Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ +TZ=UTC \ +MALLOC_MMAP_THRESHOLD_=131072 \ PERL_INLINE_DIRECTORY=/tmp/.pub-inline LimitNOFILE = 30000