From 20add372333a15adbb96b69c273f3f50e84189c7 Mon Sep 17 00:00:00 2001 From: Eric Wong Date: Wed, 11 Jan 2023 01:12:49 +0000 Subject: rewrite README and update manpage to favor CSV The CSV format has far more info, is self-describing, and Devel::Mwrap->dump never existed. I'm not sure if I care to support a Perl API now that the AF_UNIX HTTP interface exists. --- Mwrap.xs | 5 +++++ README | 61 +++++++++++++++++++++++++++++++------------------------ script/mwrap-perl | 26 ++++++++++++------------ 3 files changed, 52 insertions(+), 40 deletions(-) diff --git a/Mwrap.xs b/Mwrap.xs index 4d4c996..cbb7d15 100644 --- a/Mwrap.xs +++ b/Mwrap.xs @@ -6,6 +6,11 @@ #define MWRAP_PERL 1 #include "mwrap_core.h" +/* + * The Devel::Mwrap Perl API is probably no longer useful now that + * the AF_UNIX HTTP interface exists. It'll probably remain undocumented. + */ + /* * I hate typedefs, especially when they're hiding the fact that there's * a pointer, but XS needs this, apparently, and it does s/__/::/g diff --git a/README b/README index 1fa1dfb..88abe7d 100644 --- a/README +++ b/README @@ -4,11 +4,17 @@ Devel::Mwrap is designed to answer the question: Which lines of Perl are hitting malloc the most? -Devel::Mwrap wraps all malloc-family calls to trace the Perl source -location of such calls and bytes allocated at each callsite. It -can also function as a leak detector and show live allocations -at every call site. Depending on your application and workload, -the overhead is roughly a 50%-100% increase memory and runtime. +Devel::Mwrap wraps all malloc-family calls to trace the Perl +source location of such calls and bytes allocated at each +callsite. It is primarily designed to identify high memory use, +but may function as a leak detector as it can show live +allocations at every call site. Depending on your application +and workload, the overhead is roughly a 50%-100% increase memory +and runtime. + +It also gives configurable backtraces of all dynamically-linked +malloc callsites for any program where backtrace(3) works, +including programs not linked to Perl. It is thread-safe and requires the concurrent lock-free hash table from the Userspace RCU project: https://liburcu.org/ @@ -31,31 +37,30 @@ See `INSTALL' document == Usage -Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl script to -improve ease-of-use. You can set dump_path: in the MWRAP environment -variable to append the results to a log file: +Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl wrapper to +improve ease-of-use. You can set dump_csv: in the MWRAP environment +variable to append the results to a CSV file: - MWRAP=dump_path:/path/to/log mwrap-perl PERL_COMMAND + MWRAP=dump_csv:/path/to/csv.%p mwrap-perl COMMAND - # And to display the locations with the most allocations: - sort -k1,1rn dump, Devel::Mwrap->reset, Devel::Mwrap->each, etc. + MWRAP=socket_dir:/some/dir mwrap-perl COMMAND -However, Devel::Mwrap MUST be loaded via LD_PRELOAD to have any -effect in tracking malloc use. However, it is safe to keep -"use Devel::Mwrap" in performance-critical deployments, -as overhead is only incurred when used as an LD_PRELOAD. +And connect via `curl --unix-socket /some/dir/$PID.sock' or `mwrap-rproxy'. +See mwrap-rproxy(1p) and mwrap-perl(1p) manpages for more info. -The output of the Devel::Mwrap->dump is a text file with 3 columns: +== Compared to other tools - total_bytes call_count location +* mwrap-perl knows about Perl code, and an `mwrap' RubyGem exists, too -Where location is a Perl source location or an address retrieved -by backtrace_symbols(3). It is recommended to use the sort(1) -command on either of the first two columns to find the hottest -malloc locations. +* mwrap does not catch memory errors; use ASan, valgrind, or similar + +* mwrap is reasonably fast, fast enough for the author to run on + production-facing Perl daemons + +* the AF_UNIX HTTP interface allows inspecting live processes without + interruption instead of waiting for an exit dump == Known problems @@ -66,6 +71,8 @@ malloc locations. * Perl source files over 16.7 million lines long are not supported :P +* large C backtraces (off by default) are expensive for large programs + == Public mail archives (HTTP, Atom feeds, IMAP mailbox, NNTP group, POP3): https://80x24.org/mwrap-perl/ @@ -73,10 +80,10 @@ malloc locations. nntps://80x24.org/inbox.comp.lang.perl.mwrap https://80x24.org/mwrap-perl/_/text/help/#pop3 -No subscription nor real identities will ever be required to obtain support, -but HTML mail is rejected. Memory usage reductions start with you; -only send plain-text mail to us and do not top-post. HTML mail and -top-posting costs everybody memory and bandwidth. +No subscription nor real identities will ever be required to obtain +support or contribute, HTML mail is rejected. Memory usage reductions +start with you; only send plain-text mail to us and do not top-post. +HTML mail and top-posting costs everybody memory and bandwidth. mwrap-perl@80x24.org diff --git a/script/mwrap-perl b/script/mwrap-perl index 371aee6..cf88375 100644 --- a/script/mwrap-perl +++ b/script/mwrap-perl @@ -70,31 +70,31 @@ This may be changed via POST request (see below). Default: 0 -=item dump_path:$FILENAME +=item dump_csv:$FILENAME -Dumps the output at exit to a given filename: +Dump CSV to the given filename. - total_bytes call_count location +This output matches the HTTP server output and includes column headers, +but is subject to change in future releases. + +C without the C<:> may also be used in conjunction with +C, such as C. C<$FILENAME> may contain C<%p> where C<%p> is a placeholder for the PID being dumped. No other use of C<%> is accepted, and multiple C<%> means all C<%> (including C<%p>) are handled as-is. -=item dump_fd:$DESCRIPTOR - -As with dump_path, but dumps the output to a given file descriptor. +=item dump_path:$FILENAME -=item dump_csv:$FILENAME +Gives a simpler, legacy output compatible with the old Ruby version: -Dump CSV to the given filename. + total_bytes call_count location -This output matches the HTTP server output and includes column headers, -but is subject to change in future releases. +Expands C<%p> to the PID in C<$FILENAME> as described for C -C without the C<:> may also be used in conjunction with -C, such as C. +=item dump_fd:$DESCRIPTOR -Expands C<%p> to the PID in C<$FILENAME> as described for C +As with dump_path, but dumps the output to a given file descriptor. =back -- cgit v1.2.3-24-ge0c7