diff options
Diffstat (limited to 'README')
| -rw-r--r-- | README | 120 |
1 files changed, 70 insertions, 50 deletions
@@ -1,66 +1,86 @@ -Devel::Mwrap - LD_PRELOAD malloc wrapper + malloc line stats for Perl += mwrap - LD_PRELOAD malloc wrapper + line stats for Ruby -Devel::Mwrap is designed to answer the question: +mwrap is designed to answer the question: - Which lines of Perl are hitting malloc the most? + Which lines of Ruby are hitting malloc the most? -Devel::Mwrap wraps all malloc-family calls to trace the Perl -source location of such calls and bytes allocated at each -callsite. It is primarily designed to identify high memory use, -but may function as a leak detector as it can show live -allocations at every call site. Depending on your application -and workload, the overhead is roughly a 50%-100% increase memory -and runtime. +mwrap wraps all malloc-family calls to trace the Ruby source +location of such calls and bytes allocated at each callsite. +As of mwrap 2.0.0, it can also function as a leak detector +and show live allocations at every call site. Depending on +your application and workload, the overhead is roughly a 50-100% +increase memory and runtime with default settings. -It also gives configurable backtraces of all dynamically-linked -malloc callsites for any program where backtrace(3) works, -including programs not linked to Perl. +It works best for allocations under GVL, but tries to track +numeric caller addresses for allocations made without GVL so you +can get an idea of how much memory usage certain extensions and +native libraries use. -It is thread-safe and requires the concurrent lock-free hash table -from the Userspace RCU project: https://liburcu.org/ +As of 3.0, it also gives configurable C backtraces of all +dynamically-linked malloc callsites for any program where backtrace(3) +works, including programs not linked to Ruby. -It relies on dynamic linking to a malloc(3) implementation. If -you got Perl from your OS distribution, this typically does not -require rebuilding Perl. +It requires the concurrent lock-free hash table from the +Userspace RCU project: https://liburcu.org/ -Tested on the `perl' package distributed with: +It does not require recompiling or rebuilding Ruby, but only +supports Ruby 2.7.0 or later on a few platforms: -* Debian GNU/Linux 10 and 11 +* GNU/Linux (only tested --without-jemalloc, mwrap 3.x provides its own malloc) -* FreeBSD 12.x - -It may work on NetBSD, OpenBSD and DragonFly BSD. +It may work on FreeBSD, NetBSD, OpenBSD and DragonFly BSD if given +appropriate build options. == Install -See `INSTALL' document + # Debian-based systems: apt-get install liburcu-dev + + # Install mwrap via RubyGems.org + gem install mwrap == Usage -Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl wrapper to -improve ease-of-use. You can set dump_csv: in the MWRAP environment +mwrap works as an LD_PRELOAD and supplies a mwrap RubyGem executable to +improve ease-of-use. You can set `dump_csv:' in the MWRAP environment variable to append the results to a CSV file: - MWRAP=dump_csv:/path/to/csv.%p mwrap-perl COMMAND + MWRAP=dump_csv:/path/to/log mwrap RUBY_COMMAND + +(`dump_csv:' is new in mwrap 3.x, `dump_file:' from earlier versions is +still supported). For long running processes, you can see the AF_UNIX HTTP interface: - MWRAP=socket_dir:/some/dir mwrap-perl COMMAND + MWRAP=socket_dir:/some/dir mwrap COMMAND + +And connect via `curl --unix-socket /some/dir/$PID.sock' or +`mwrap-rproxy(1p)<https://80x24.org/mwrap-perl.git/tree/script/mwrap-rproxy#n44> +for more info. -And connect via `curl --unix-socket /some/dir/$PID.sock' or `mwrap-rproxy'. -See mwrap-rproxy(1p) and mwrap-perl(1p) manpages for more info. +You may also `require "mwrap"' in your Ruby code and use +Mwrap.dump, Mwrap.reset, Mwrap.each, etc. -== Compared to other tools +However, mwrap MUST be loaded via LD_PRELOAD to have any +effect in tracking malloc use. However, it is safe to keep +"require 'mwrap'" in performance-critical deployments, +as overhead is only incurred when used as an LD_PRELOAD. -* mwrap-perl knows about Perl code, and an `mwrap' RubyGem exists, too +The output of `dump_csv:' is has self-describing columns and is +subject to change. SQLite 3.32+ can load it with: `.import --csv'. -* mwrap does not catch memory errors; use ASan, valgrind, or similar +The output of the `dump_file:' output is a text file with 3 columns: -* mwrap is reasonably fast, fast enough for the author to run on - production-facing Perl daemons + total_bytes call_count location -* the AF_UNIX HTTP interface allows inspecting live processes without - interruption instead of waiting for an exit dump +Where location is a Ruby source location (if made under GVL) +or an address retrieved by backtrace_symbols(3). It is +recommended to use the sort(1) command on either of the +first two columns to find the hottest malloc locations. + +mwrap 3.0.0+ also supports an embedded HTTP server +it is documented at: + +https://80x24.org/mwrap.git/tree/Documentation/mwrap.pod == Known problems @@ -68,28 +88,28 @@ See mwrap-rproxy(1p) and mwrap-perl(1p) manpages for more info. * signalfd(2)-reliant code will need latest URCU with commit ea3a28a3f71dd02f (Disable signals in URCU background threads, 2022-09-23) + This doesn't affect C Ruby itself, and signalfd(2) use is rare + 3rd-party processes. -* Perl source files over 16.7 million lines long are not supported :P - -* large C backtraces (off by default) are expensive for large programs +* Ruby source files over 16.7 million lines long are not supported :P == Public mail archives (HTTP, Atom feeds, IMAP mailbox, NNTP group, POP3): - https://80x24.org/mwrap-perl/ - imaps://;AUTH=ANONYMOUS@80x24.org/inbox.comp.lang.perl.mwrap.0 - nntps://80x24.org/inbox.comp.lang.perl.mwrap - https://80x24.org/mwrap-perl/_/text/help/#pop3 +* https://80x24.org/mwrap-public/ +* nntps://80x24.org/inbox.comp.lang.ruby.mwrap +* imaps://;AUTH=ANONYMOUS@80x24.org/inbox.comp.lang.ruby.mwrap.0 +* https://80x24.org/mwrap-public/_/text/help/#pop3 -No subscription nor real identities will ever be required to obtain -support or contribute, HTML mail is rejected. Memory usage reductions -start with you; only send plain-text mail to us and do not top-post. -HTML mail and top-posting costs everybody memory and bandwidth. +No subscription nor real identities will ever be required to obtain support, +but HTML mail is rejected. Memory usage reductions start with you; +only send plain-text mail to us and do not top-post. HTML mail and +top-posting costs everybody memory and bandwidth. - mwrap-perl@80x24.org + mwrap-public@80x24.org == Hacking - git clone https://80x24.org/mwrap-perl.git + git clone https://80x24.org/mwrap.git Send all patches ("git format-patch" + "git send-email") and pull requests (use "git request-pull" to format) via email |