* [PATCH 0/4] CSV-related improvements
@ 2023-01-11 1:12 7% Eric Wong
2023-01-11 1:12 6% ` [PATCH 4/4] rewrite README and update manpage to favor CSV Eric Wong
0 siblings, 1 reply; 2+ results
From: Eric Wong @ 2023-01-11 1:12 UTC (permalink / raw)
To: mwrap-perl
It should've been CSV from the beginning...
Eric Wong (4):
support MWRAP=dump_csv:$FILENAME parameter
add mwrap-decode-csv tool
%p => PID expansion for dump_path + dump_csv
rewrite README and update manpage to favor CSV
MANIFEST | 1 +
Makefile.PL | 3 +-
Mwrap.xs | 5 +
README | 61 ++++++-----
httpd.h | 217 ++++++++++++++++++++------------------
lib/Devel/Mwrap/Rproxy.pm | 33 +++---
mwrap_core.h | 50 +++++++--
script/mwrap-decode-csv | 57 ++++++++++
script/mwrap-perl | 18 +++-
t/mwrap.t | 36 ++++++-
10 files changed, 327 insertions(+), 154 deletions(-)
create mode 100644 script/mwrap-decode-csv
^ permalink raw reply [relevance 7%]
* [PATCH 4/4] rewrite README and update manpage to favor CSV
2023-01-11 1:12 7% [PATCH 0/4] CSV-related improvements Eric Wong
@ 2023-01-11 1:12 6% ` Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2023-01-11 1:12 UTC (permalink / raw)
To: mwrap-perl
The CSV format has far more info, is self-describing, and
Devel::Mwrap->dump never existed. I'm not sure if I care to
support a Perl API now that the AF_UNIX HTTP interface exists.
---
Mwrap.xs | 5 ++++
README | 61 ++++++++++++++++++++++++++---------------------
script/mwrap-perl | 26 ++++++++++----------
3 files changed, 52 insertions(+), 40 deletions(-)
diff --git a/Mwrap.xs b/Mwrap.xs
index 4d4c996..cbb7d15 100644
--- a/Mwrap.xs
+++ b/Mwrap.xs
@@ -6,6 +6,11 @@
#define MWRAP_PERL 1
#include "mwrap_core.h"
+/*
+ * The Devel::Mwrap Perl API is probably no longer useful now that
+ * the AF_UNIX HTTP interface exists. It'll probably remain undocumented.
+ */
+
/*
* I hate typedefs, especially when they're hiding the fact that there's
* a pointer, but XS needs this, apparently, and it does s/__/::/g
diff --git a/README b/README
index 1fa1dfb..88abe7d 100644
--- a/README
+++ b/README
@@ -4,11 +4,17 @@ Devel::Mwrap is designed to answer the question:
Which lines of Perl are hitting malloc the most?
-Devel::Mwrap wraps all malloc-family calls to trace the Perl source
-location of such calls and bytes allocated at each callsite. It
-can also function as a leak detector and show live allocations
-at every call site. Depending on your application and workload,
-the overhead is roughly a 50%-100% increase memory and runtime.
+Devel::Mwrap wraps all malloc-family calls to trace the Perl
+source location of such calls and bytes allocated at each
+callsite. It is primarily designed to identify high memory use,
+but may function as a leak detector as it can show live
+allocations at every call site. Depending on your application
+and workload, the overhead is roughly a 50%-100% increase memory
+and runtime.
+
+It also gives configurable backtraces of all dynamically-linked
+malloc callsites for any program where backtrace(3) works,
+including programs not linked to Perl.
It is thread-safe and requires the concurrent lock-free hash table
from the Userspace RCU project: https://liburcu.org/
@@ -31,31 +37,30 @@ See `INSTALL' document
== Usage
-Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl script to
-improve ease-of-use. You can set dump_path: in the MWRAP environment
-variable to append the results to a log file:
+Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl wrapper to
+improve ease-of-use. You can set dump_csv: in the MWRAP environment
+variable to append the results to a CSV file:
- MWRAP=dump_path:/path/to/log mwrap-perl PERL_COMMAND
+ MWRAP=dump_csv:/path/to/csv.%p mwrap-perl COMMAND
- # And to display the locations with the most allocations:
- sort -k1,1rn </path/to/log | $PAGER
+For long running processes, you can see the AF_UNIX HTTP interface:
-You may also `use Devel::Mwrap' in your Perl code and use
-Devel::Mwrap->dump, Devel::Mwrap->reset, Devel::Mwrap->each, etc.
+ MWRAP=socket_dir:/some/dir mwrap-perl COMMAND
-However, Devel::Mwrap MUST be loaded via LD_PRELOAD to have any
-effect in tracking malloc use. However, it is safe to keep
-"use Devel::Mwrap" in performance-critical deployments,
-as overhead is only incurred when used as an LD_PRELOAD.
+And connect via `curl --unix-socket /some/dir/$PID.sock' or `mwrap-rproxy'.
+See mwrap-rproxy(1p) and mwrap-perl(1p) manpages for more info.
-The output of the Devel::Mwrap->dump is a text file with 3 columns:
+== Compared to other tools
- total_bytes call_count location
+* mwrap-perl knows about Perl code, and an `mwrap' RubyGem exists, too
-Where location is a Perl source location or an address retrieved
-by backtrace_symbols(3). It is recommended to use the sort(1)
-command on either of the first two columns to find the hottest
-malloc locations.
+* mwrap does not catch memory errors; use ASan, valgrind, or similar
+
+* mwrap is reasonably fast, fast enough for the author to run on
+ production-facing Perl daemons
+
+* the AF_UNIX HTTP interface allows inspecting live processes without
+ interruption instead of waiting for an exit dump
== Known problems
@@ -66,6 +71,8 @@ malloc locations.
* Perl source files over 16.7 million lines long are not supported :P
+* large C backtraces (off by default) are expensive for large programs
+
== Public mail archives (HTTP, Atom feeds, IMAP mailbox, NNTP group, POP3):
https://80x24.org/mwrap-perl/
@@ -73,10 +80,10 @@ malloc locations.
nntps://80x24.org/inbox.comp.lang.perl.mwrap
https://80x24.org/mwrap-perl/_/text/help/#pop3
-No subscription nor real identities will ever be required to obtain support,
-but HTML mail is rejected. Memory usage reductions start with you;
-only send plain-text mail to us and do not top-post. HTML mail and
-top-posting costs everybody memory and bandwidth.
+No subscription nor real identities will ever be required to obtain
+support or contribute, HTML mail is rejected. Memory usage reductions
+start with you; only send plain-text mail to us and do not top-post.
+HTML mail and top-posting costs everybody memory and bandwidth.
mwrap-perl@80x24.org
diff --git a/script/mwrap-perl b/script/mwrap-perl
index 371aee6..cf88375 100644
--- a/script/mwrap-perl
+++ b/script/mwrap-perl
@@ -70,31 +70,31 @@ This may be changed via POST request (see below).
Default: 0
-=item dump_path:$FILENAME
+=item dump_csv:$FILENAME
-Dumps the output at exit to a given filename:
+Dump CSV to the given filename.
- total_bytes call_count location
+This output matches the HTTP server output and includes column headers,
+but is subject to change in future releases.
+
+C<dump_csv> without the C<:> may also be used in conjunction with
+C<dump_fd>, such as C<MWRAP=dump_fd:2,dump_csv>.
C<$FILENAME> may contain C<%p> where C<%p> is a placeholder for
the PID being dumped. No other use of C<%> is accepted, and
multiple C<%> means all C<%> (including C<%p>) are handled as-is.
-=item dump_fd:$DESCRIPTOR
-
-As with dump_path, but dumps the output to a given file descriptor.
+=item dump_path:$FILENAME
-=item dump_csv:$FILENAME
+Gives a simpler, legacy output compatible with the old Ruby version:
-Dump CSV to the given filename.
+ total_bytes call_count location
-This output matches the HTTP server output and includes column headers,
-but is subject to change in future releases.
+Expands C<%p> to the PID in C<$FILENAME> as described for C<dump_csv>
-C<dump_csv> without the C<:> may also be used in conjunction with
-C<dump_fd>, such as C<MWRAP=dump_fd:2,dump_csv>.
+=item dump_fd:$DESCRIPTOR
-Expands C<%p> to the PID in C<$FILENAME> as described for C<dump_path:>
+As with dump_path, but dumps the output to a given file descriptor.
=back
^ permalink raw reply related [relevance 6%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2023-01-11 1:12 7% [PATCH 0/4] CSV-related improvements Eric Wong
2023-01-11 1:12 6% ` [PATCH 4/4] rewrite README and update manpage to favor CSV Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/mwrap-perl.git
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).