From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: X-Spam-Status: No, score=-4.0 required=3.0 tests=ALL_TRUSTED,AWL,BAYES_00 shortcircuit=no autolearn=ham autolearn_force=no version=3.4.1 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id 1DCA51F597; Mon, 16 Jul 2018 21:48:40 +0000 (UTC) Date: Mon, 16 Jul 2018 21:48:40 +0000 From: Eric Wong To: Sam Saffron Cc: ruby-talk@ruby-lang.org, mwrap-public@80x24.org Subject: Re: [ANN] mwrap - LD_PRELOAD malloc wrapper + line stats for Ruby Message-ID: <20180716214840.GA8819@dcvr> References: <20180702120050.GA24029@dcvr> <20180703074824.GA22835@dcvr> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180703074824.GA22835@dcvr> List-Id: Eric Wong wrote: > Sam Saffron wrote: > > Is there any chance we can have a mode with "free" tracking as well? > > That way we can enable this on a long running process to detect leaks? > > Maybe, but it might get a lot more expensive to be useful... > Merely counting frees and associating them with source lines > like mwrap currently does with *allocs would not be useful, > since GC can call free from just about anywhere. Definitely more expensive, but it's still usable: https://80x24.org/mwrap-public/20180716211933.5835-1-e@80x24.org/ TL;DR: live demo of the new features running inside a Rack app: https://80x24.org/MWRAP/each/2000 The following changes since commit 834de3bc0da4af53535d5c9d4975e546df9fb186: bin/mwrap: support LISTEN_FDS env from systemd (2018-07-16 19:33:12 +0000) are available in the Git repository at: https://80x24.org/mwrap.git heavy for you to fetch changes up to c432e3ad30aa247dbac8575af87b0c594365d3fd: Cloning from that git URL runs it through the same process which is running /MWRAP/. (I will eventually replace cgit on 80x24.org with repobrowse ) > So we'd have to try per-allocation accounting to know for sure > and it could get time consuming. So depending on the application, it's up to 50% more memory intensive and 50% slower. Though likely cheaper than most leak checkers (definitely cheaper than valgrind's leakchecker). memalign functions (used by CoW-friendly Ruby GC) ends up causing massive fragmentation as a result, because we can't reuse the the overallocated part like a full malloc implementation can. And I even disabled tracking allocations made by them by default, since it can be noisy and misleading when hit in cold paths. > Yes, if we add per-allocation tracking, storing the > rb_gc_count() result for each malloc count could be useful to > track the relative age of each malloc-ed region. Yes, that's done, and the /at/$LOCATION endpoint lets us see the size and rb_gc_count value at the time of allocation for each live allocation: https://80x24.org/MWRAP/at//home/rb/lib/ruby/gems/2.6.0/gems/rack-2.0.5/lib/rack/deflater.rb:77 (note the double slash :<)