From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id F19B21F597; Fri, 20 Jul 2018 09:25:16 +0000 (UTC) Date: Fri, 20 Jul 2018 09:25:16 +0000 From: Eric Wong To: ruby-talk@ruby-lang.org, mwrap-public@80x24.org Subject: [PATCH] mwrap 2.0.0 mwrap - LD_PRELOAD malloc wrapper for Ruby Message-ID: <20180720092516.GA23759@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline List-Id: mwrap is designed to answer the question: Which lines of Ruby are hitting malloc the most? mwrap wraps all malloc-family calls to trace the Ruby source location of such calls and bytes allocated at each callsite. As of mwrap 2.0.0, it can also function as a leak detector and show live allocations at every call site. Depending on your application and workload, the overhead is roughly a 50% increase memory and runtime. It works best for allocations under GVL, but tries to track numeric caller addresses for allocations made without GVL so you can get an idea of how much memory usage certain extensions and native libraries use. It requires the concurrent lock-free hash table from the Userspace RCU project: https://liburcu.org/ It does not require recompiling or rebuilding Ruby, but only supports Ruby trunk (2.6.0dev+) on a few platforms: * GNU/Linux * FreeBSD (tested 11.1) It may work on NetBSD, OpenBSD and DragonFly BSD. Changes in 2.0.0: This release includes significant changes to track live allocations and frees. It can find memory leaks from malloc with less overhead than valgrind's leakchecker and there is a new Rack endpoint (MwrapRack) which can display live allocation stats. API additions: * Mwrap#[] - https://80x24.org/mwrap/Mwrap.html#method-c-5B-5D * Mwrap::SourceLocation - https://80x24.org/mwrap/Mwrap/SourceLocation.html * MwrapRack - https://80x24.org/mwrap/MwrapRack.html Incompatible changes: * Mwrap.clear now an alias to Mwrap.reset; as it's unsafe to implement the new Mwrap#[] API otherwise: https://80x24.org/mwrap-public/20180716211933.5835-12-e@80x24.org/ 26 changes since v1.0.0: README: improve usage example MANIFEST: add .document add benchmark use __attribute__((weak)) instead of dlsym Mwrap.dump: do not segfault on invalid IO arg bin/mwrap: support LISTEN_FDS env from systemd support per-allocation headers for per-alloc tracking mwrap: use malloc to do our own memalign hold RCU read lock to insert each allocation realloc: do not copy if allocation failed internal_memalign: do not assume real_malloc succeeds ensure ENOMEM is preserved in errno when appropriate memalign: check alignment on all public functions reduce stack usage from file names resolve real_malloc earlier for C++ programs allow analyzing live allocations via Mwrap[location] alias Mwrap.clear to Mwrap.reset implement accessors for SourceLocation mwrap_aref: quiet -Wshorten-64-to-32 warning fixes for FreeBSD 11.1... use memrchr to extract address under glibc do not track allocations for constructor and Init_ disable memalign tracking by default support Mwrap.quiet to temporarily disable allocation tracking mwrap_rack: Rack app to track live allocations documentation updates for 2.0.0 release