mwrap user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: Sam Saffron <sam.saffron@gmail.com>
Cc: ruby-talk@ruby-lang.org, mwrap-public@80x24.org
Subject: Re: [ANN] mwrap - LD_PRELOAD malloc wrapper + line stats for Ruby
Date: Mon, 16 Jul 2018 21:48:40 +0000	[thread overview]
Message-ID: <20180716214840.GA8819@dcvr> (raw)
In-Reply-To: <20180703074824.GA22835@dcvr>

Eric Wong <e@80x24.org> wrote:
> Sam Saffron <sam.saffron@gmail.com> wrote:
> > Is there any chance we can have a mode with "free" tracking as well?
> > That way we can enable this on a long running process to detect leaks?
> 
> Maybe, but it might get a lot more expensive to be useful...
> Merely counting frees and associating them with source lines
> like mwrap currently does with *allocs would not be useful,
> since GC can call free from just about anywhere.

Definitely more expensive, but it's still usable:

   https://80x24.org/mwrap-public/20180716211933.5835-1-e@80x24.org/

TL;DR: live demo of the new features running inside a Rack app:

  https://80x24.org/MWRAP/each/2000

The following changes since commit 834de3bc0da4af53535d5c9d4975e546df9fb186:

  bin/mwrap: support LISTEN_FDS env from systemd (2018-07-16 19:33:12 +0000)

are available in the Git repository at:

  https://80x24.org/mwrap.git heavy

for you to fetch changes up to c432e3ad30aa247dbac8575af87b0c594365d3fd:

Cloning from that git URL runs it through the same process which
is running /MWRAP/.  (I will eventually replace cgit on
80x24.org with repobrowse <https://80x24.org/repobrowse/README>)

> So we'd have to try per-allocation accounting to know for sure
> and it could get time consuming.

So depending on the application, it's up to 50% more memory
intensive and 50% slower.  Though likely cheaper than most leak
checkers (definitely cheaper than valgrind's leakchecker).

memalign functions (used by CoW-friendly Ruby GC) ends up
causing massive fragmentation as a result, because we can't reuse
the the overallocated part like a full malloc implementation
can.  And I even disabled tracking allocations made by them by
default, since it can be noisy and misleading when hit in cold
paths.

> Yes, if we add per-allocation tracking, storing the
> rb_gc_count() result for each malloc count could be useful to
> track the relative age of each malloc-ed region.

Yes, that's done, and the /at/$LOCATION endpoint lets us see
the size and rb_gc_count value at the time of allocation
for each live allocation:

https://80x24.org/MWRAP/at//home/rb/lib/ruby/gems/2.6.0/gems/rack-2.0.5/lib/rack/deflater.rb:77

(note the double slash :<)

  reply	other threads:[~2018-07-16 21:48 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-07-02 12:00 [ANN] mwrap - LD_PRELOAD malloc wrapper + line stats for Ruby Eric Wong
2018-07-03  0:11 ` Sam Saffron
2018-07-03  7:48   ` Eric Wong
2018-07-16 21:48     ` Eric Wong [this message]
2018-07-03  9:11 ` Eric Wong

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

  List information: https://80x24.org/mwrap/

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180716214840.GA8819@dcvr \
    --to=e@80x24.org \
    --cc=mwrap-public@80x24.org \
    --cc=ruby-talk@ruby-lang.org \
    --cc=sam.saffron@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mwrap.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).