From 298b05cef615ae3d3f1323e805fe135ae5138144 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Mon, 11 Mar 2024 19:40:11 +0000 Subject: doc: tuning: note reduced fragmentation w/ jemalloc I may be mistaken, but I suspect the reason jemalloc handles long-lived processes better than glibc is due to granularity reduction being scaled to larger size classes. This can waste 20% of an individual allocation, but increases the likelyhood of reuse (without splitting/consolidating into other sizes). In other words, glibc seems to try too hard to make the best fit for initial allocations. This ends up being suboptimal over time as those allocations are freed and similar (but not identical) allocations come in. jemalloc sacrifices the best initial fit for better fits over a long process lifetime. --- Documentation/public-inbox-tuning.pod | 5 +++++ examples/public-inbox-netd@.service | 2 ++ 2 files changed, 7 insertions(+) diff --git a/Documentation/public-inbox-tuning.pod b/Documentation/public-inbox-tuning.pod index 38810ce6..73246144 100644 --- a/Documentation/public-inbox-tuning.pod +++ b/Documentation/public-inbox-tuning.pod @@ -163,6 +163,11 @@ Transport Layer Security (IMAPS, NNTPS, or via STARTTLS) significantly increases memory use of client sockets, be sure to account for that in capacity planning. +Bursts of small object allocations late in process life contribute to +fragmentation of the heap due to arenas (slabs) used internally by Perl. +jemalloc (tested as an LD_PRELOAD on GNU/Linux) appears to reduce +overall fragmentation compared to glibc malloc in long-lived processes. + =head2 Other OS tuning knobs Linux users: the C sysctl may need to be increased if diff --git a/examples/public-inbox-netd@.service b/examples/public-inbox-netd@.service index de5feea6..2330bd59 100644 --- a/examples/public-inbox-netd@.service +++ b/examples/public-inbox-netd@.service @@ -12,6 +12,8 @@ Wants = public-inbox-netd.socket After = public-inbox-netd.socket [Service] +# An LD_PRELOAD for libjemalloc can be added here. It currently seems +# more resistant to fragmentation to glibc in long-lived daemons. Environment = PI_CONFIG=/home/pi/.public-inbox/config \ PATH=/usr/local/bin:/usr/bin:/bin \ TZ=UTC \ -- cgit v1.2.3-24-ge0c7