mwrap (Perl version) user+dev discussion/patches/pulls/bugs/help
 help / color / mirror / code / Atom feed
From: Eric Wong <e@80x24.org>
To: mwrap-perl@80x24.org
Subject: [PATCH 4/4] rewrite README and update manpage to favor CSV
Date: Wed, 11 Jan 2023 01:12:49 +0000	[thread overview]
Message-ID: <20230111011249.2713039-5-e@80x24.org> (raw)
In-Reply-To: <20230111011249.2713039-1-e@80x24.org>

The CSV format has far more info, is self-describing, and
Devel::Mwrap->dump never existed.  I'm not sure if I care to
support a Perl API now that the AF_UNIX HTTP interface exists.
---
 Mwrap.xs          |  5 ++++
 README            | 61 ++++++++++++++++++++++++++---------------------
 script/mwrap-perl | 26 ++++++++++----------
 3 files changed, 52 insertions(+), 40 deletions(-)

diff --git a/Mwrap.xs b/Mwrap.xs
index 4d4c996..cbb7d15 100644
--- a/Mwrap.xs
+++ b/Mwrap.xs
@@ -6,6 +6,11 @@
 #define MWRAP_PERL 1
 #include "mwrap_core.h"
 
+/*
+ * The Devel::Mwrap Perl API is probably no longer useful now that
+ * the AF_UNIX HTTP interface exists.  It'll probably remain undocumented.
+ */
+
 /*
  * I hate typedefs, especially when they're hiding the fact that there's
  * a pointer, but XS needs this, apparently, and it does s/__/::/g
diff --git a/README b/README
index 1fa1dfb..88abe7d 100644
--- a/README
+++ b/README
@@ -4,11 +4,17 @@ Devel::Mwrap is designed to answer the question:
 
    Which lines of Perl are hitting malloc the most?
 
-Devel::Mwrap wraps all malloc-family calls to trace the Perl source
-location of such calls and bytes allocated at each callsite.  It
-can also function as a leak detector and show live allocations
-at every call site.  Depending on your application and workload,
-the overhead is roughly a 50%-100% increase memory and runtime.
+Devel::Mwrap wraps all malloc-family calls to trace the Perl
+source location of such calls and bytes allocated at each
+callsite.  It is primarily designed to identify high memory use,
+but may function as a leak detector as it can show live
+allocations at every call site.  Depending on your application
+and workload, the overhead is roughly a 50%-100% increase memory
+and runtime.
+
+It also gives configurable backtraces of all dynamically-linked
+malloc callsites for any program where backtrace(3) works,
+including programs not linked to Perl.
 
 It is thread-safe and requires the concurrent lock-free hash table
 from the Userspace RCU project: https://liburcu.org/
@@ -31,31 +37,30 @@ See `INSTALL' document
 
 == Usage
 
-Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl script to
-improve ease-of-use.  You can set dump_path: in the MWRAP environment
-variable to append the results to a log file:
+Devel::Mwrap works as an LD_PRELOAD and supplies a mwrap-perl wrapper to
+improve ease-of-use.  You can set dump_csv: in the MWRAP environment
+variable to append the results to a CSV file:
 
-	MWRAP=dump_path:/path/to/log mwrap-perl PERL_COMMAND
+	MWRAP=dump_csv:/path/to/csv.%p mwrap-perl COMMAND
 
-	# And to display the locations with the most allocations:
-	sort -k1,1rn </path/to/log | $PAGER
+For long running processes, you can see the AF_UNIX HTTP interface:
 
-You may also `use Devel::Mwrap' in your Perl code and use
-Devel::Mwrap->dump, Devel::Mwrap->reset, Devel::Mwrap->each, etc.
+	MWRAP=socket_dir:/some/dir mwrap-perl COMMAND
 
-However, Devel::Mwrap MUST be loaded via LD_PRELOAD to have any
-effect in tracking malloc use.  However, it is safe to keep
-"use Devel::Mwrap" in performance-critical deployments,
-as overhead is only incurred when used as an LD_PRELOAD.
+And connect via `curl --unix-socket /some/dir/$PID.sock' or `mwrap-rproxy'.
+See mwrap-rproxy(1p) and mwrap-perl(1p) manpages for more info.
 
-The output of the Devel::Mwrap->dump is a text file with 3 columns:
+== Compared to other tools
 
-	total_bytes	call_count	location
+* mwrap-perl knows about Perl code, and an `mwrap' RubyGem exists, too
 
-Where location is a Perl source location or an address retrieved
-by backtrace_symbols(3).  It is recommended to use the sort(1)
-command on either of the first two columns to find the hottest
-malloc locations.
+* mwrap does not catch memory errors; use ASan, valgrind, or similar
+
+* mwrap is reasonably fast, fast enough for the author to run on
+  production-facing Perl daemons
+
+* the AF_UNIX HTTP interface allows inspecting live processes without
+  interruption instead of waiting for an exit dump
 
 == Known problems
 
@@ -66,6 +71,8 @@ malloc locations.
 
 * Perl source files over 16.7 million lines long are not supported :P
 
+* large C backtraces (off by default) are expensive for large programs
+
 == Public mail archives (HTTP, Atom feeds, IMAP mailbox, NNTP group, POP3):
 
 	https://80x24.org/mwrap-perl/
@@ -73,10 +80,10 @@ malloc locations.
 	nntps://80x24.org/inbox.comp.lang.perl.mwrap
 	https://80x24.org/mwrap-perl/_/text/help/#pop3
 
-No subscription nor real identities will ever be required to obtain support,
-but HTML mail is rejected.  Memory usage reductions start with you;
-only send plain-text mail to us and do not top-post.  HTML mail and
-top-posting costs everybody memory and bandwidth.
+No subscription nor real identities will ever be required to obtain
+support or contribute, HTML mail is rejected.  Memory usage reductions
+start with you; only send plain-text mail to us and do not top-post.
+HTML mail and top-posting costs everybody memory and bandwidth.
 
 		mwrap-perl@80x24.org
 
diff --git a/script/mwrap-perl b/script/mwrap-perl
index 371aee6..cf88375 100644
--- a/script/mwrap-perl
+++ b/script/mwrap-perl
@@ -70,31 +70,31 @@ This may be changed via POST request (see below).
 
 Default: 0
 
-=item dump_path:$FILENAME
+=item dump_csv:$FILENAME
 
-Dumps the output at exit to a given filename:
+Dump CSV to the given filename.
 
-	total_bytes	call_count	location
+This output matches the HTTP server output and includes column headers,
+but is subject to change in future releases.
+
+C<dump_csv> without the C<:> may also be used in conjunction with
+C<dump_fd>, such as C<MWRAP=dump_fd:2,dump_csv>.
 
 C<$FILENAME> may contain C<%p> where C<%p> is a placeholder for
 the PID being dumped.  No other use of C<%> is accepted, and
 multiple C<%> means all C<%> (including C<%p>) are handled as-is.
 
-=item dump_fd:$DESCRIPTOR
-
-As with dump_path, but dumps the output to a given file descriptor.
+=item dump_path:$FILENAME
 
-=item dump_csv:$FILENAME
+Gives a simpler, legacy output compatible with the old Ruby version:
 
-Dump CSV to the given filename.
+	total_bytes	call_count	location
 
-This output matches the HTTP server output and includes column headers,
-but is subject to change in future releases.
+Expands C<%p> to the PID in C<$FILENAME> as described for C<dump_csv>
 
-C<dump_csv> without the C<:> may also be used in conjunction with
-C<dump_fd>, such as C<MWRAP=dump_fd:2,dump_csv>.
+=item dump_fd:$DESCRIPTOR
 
-Expands C<%p> to the PID in C<$FILENAME> as described for C<dump_path:>
+As with dump_path, but dumps the output to a given file descriptor.
 
 =back
 

      parent reply	other threads:[~2023-01-11  1:12 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-01-11  1:12 [PATCH 0/4] CSV-related improvements Eric Wong
2023-01-11  1:12 ` [PATCH 1/4] support MWRAP=dump_csv:$FILENAME parameter Eric Wong
2023-01-11  1:12 ` [PATCH 2/4] add mwrap-decode-csv tool Eric Wong
2023-01-11  1:12 ` [PATCH 3/4] %p => PID expansion for dump_path + dump_csv Eric Wong
2023-01-11  1:12 ` Eric Wong [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230111011249.2713039-5-e@80x24.org \
    --to=e@80x24.org \
    --cc=mwrap-perl@80x24.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
Code repositories for project(s) associated with this public inbox

	https://80x24.org/mwrap-perl.git

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).