* [PATCH] various doc updates, add mwrap(1) manpage
@ 2023-01-09 5:52 2% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2023-01-09 5:52 UTC (permalink / raw)
To: mwrap-public
FreeBSD 12.x doesn't seem to work with the `pkg install'-ed
ruby
---
Documentation/.gitignore | 2 +
Documentation/GNUmakefile | 63 +++++++++++++++++++
Documentation/mwrap.pod | 123 ++++++++++++++++++++++++++++++++++++++
MANIFEST | 4 +-
README | 38 +++++++-----
mwrap.gemspec | 11 +++-
6 files changed, 223 insertions(+), 18 deletions(-)
create mode 100644 Documentation/.gitignore
create mode 100644 Documentation/GNUmakefile
create mode 100644 Documentation/mwrap.pod
diff --git a/Documentation/.gitignore b/Documentation/.gitignore
new file mode 100644
index 0000000..1b0e502
--- /dev/null
+++ b/Documentation/.gitignore
@@ -0,0 +1,2 @@
+mwrap.txt
+mwrap.1
diff --git a/Documentation/GNUmakefile b/Documentation/GNUmakefile
new file mode 100644
index 0000000..14480da
--- /dev/null
+++ b/Documentation/GNUmakefile
@@ -0,0 +1,63 @@
+# Copyright (C) all contributors <mwrap-public@80x24.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+all::
+
+INSTALL = install
+POD2MAN = pod2man
+VERSION := $(shell cd .. && ./VERSION-GEN)
+release := mwrap $(VERSION)
+POD2MAN_OPTS = -v -r '$(release)' --stderr -d 1993-10-02 -c 'mwrap user manual'
+pod2man = $(POD2MAN) $(POD2MAN_OPTS)
+POD2TEXT = pod2text
+POD2TEXT_OPTS = --stderr
+pod2text = $(POD2TEXT) $(POD2TEXT_OPTS)
+
+m1 = mwrap
+m5 =
+m7 =
+
+man1 := $(addsuffix .1, $(m1))
+man5 := $(addsuffix .5, $(m5))
+man7 := $(addsuffix .7, $(m7))
+
+all:: man
+
+man: $(man1) $(man5) $(man7)
+
+prefix ?= $(HOME)
+mandir ?= $(prefix)/share/man
+man1dir = $(mandir)/man1
+man5dir = $(mandir)/man5
+man7dir = $(mandir)/man7
+
+gem-man: man
+ $(INSTALL) -d -m 755 ../man
+ test -z "$(man1)" || $(INSTALL) -m 644 $(man1) ../man
+ test -z "$(man5)" || $(INSTALL) -m 644 $(man5) ../man
+ test -z "$(man7)" || $(INSTALL) -m 644 $(man7) ../man
+
+install-man: man
+ $(INSTALL) -d -m 755 $(DESTDIR)$(man1dir)
+ $(INSTALL) -d -m 755 $(DESTDIR)$(man5dir)
+ test -z "$(man7)" || $(INSTALL) -d -m 755 $(DESTDIR)$(man7dir)
+ $(INSTALL) -m 644 $(man1) $(DESTDIR)$(man1dir)
+ $(INSTALL) -m 644 $(man5) $(DESTDIR)$(man5dir)
+ test -z "$(man7)" || $(INSTALL) -m 644 $(man7) $(DESTDIR)$(man7dir)
+
+%.1 %.5 %.7 : %.pod
+ $(pod2man) -s $(subst .,,$(suffix $@)) $< $@+ && mv $@+ $@
+
+mantxt = $(addsuffix .txt, $(m1) $(m5) $(m7))
+
+txt :: $(mantxt)
+
+all :: txt
+
+%.txt : %.pod
+ $(pod2text) $< $@+
+ touch -r $< $@+
+ mv $@+ $@
+
+clean::
+ $(RM) $(man1) $(man5) $(man7)
+ $(RM) $(addsuffix .txt.gz, $(m1) $(m5) $(m7))
diff --git a/Documentation/mwrap.pod b/Documentation/mwrap.pod
new file mode 100644
index 0000000..6832430
--- /dev/null
+++ b/Documentation/mwrap.pod
@@ -0,0 +1,123 @@
+=head1 NAME
+
+mwrap - run any command under mwrap
+
+=head1 SYNOPSIS
+
+ # to trace a long-running program and access it via $DIR/$PID.sock:
+ MWRAP=socket_dir:$DIR mwrap COMMAND
+
+ # to trace a short-lived command and dump its output to a log:
+ MWRAP=dump_path:$FILENAME mwrap COMMAND
+
+=head1 DESCRIPTION
+
+mwrap is a command-line to automatically add mwrap.so as an
+LD_PRELOAD for any command. It will resolve malloc-family calls
+to a Ruby file and line number, and it can also provide a backtrace
+of native (C/C++) functions for non-Ruby programs.
+
+=head1 ENVIRONMENT
+
+C<MWRAP> is the only environment variable read. It contains multiple
+options delimited by C<,> with names and values delimited by C<:>
+
+=over 4
+
+=item socket_dir:$DIR
+
+This launches an embedded HTTP server in each process and binds it
+to C<$DIR/$PID.sock>. C<curl --unix-socket $DIR/$PID.sock>
+or L<mwrap-rproxy(1p)> (from L<https://80x24.org/mwrap-perl.git>) may
+be used to access various endpoints in the HTTP server.
+
+=item bt:$DEPTH
+
+The backtrace depth for L<backtrace(3)> in addition to the Perl
+file and line number where C<$DEPTH> is a non-negative number.
+
+The maximum allowed value is 32, though values of 5 or less are
+typically useful. Increasing this to even 2 or 3 can significantly
+increase the amount of memory mwrap (and liburcu) itself uses.
+
+This is only useful in conjunction with C<socket_dir>
+
+This may be changed via POST request (see below).
+
+Default: 0
+
+=item dump_path:$FILENAME
+
+Dumps the output at exit to a given filename:
+
+ total_bytes call_count location
+
+In the future, dumping to a self-describing CSV will be supported.
+
+=item dump_fd:$DESCRIPTOR
+
+As with dump_path, but dumps the output to a given file descriptor.
+
+=back
+
+=head1 HTTP POST API
+
+In addition to the various GET endpoints linked via C<http://0/$PID/>,
+there are some POST endpoints which are typically accessed via
+C<curl --unix-socket $DIR/$PID.sock>
+
+=over 4
+
+=item POST http://0/$PID/reset
+
+C<curl --unix-socket $DIR/$PID.sock -XPOST http://0/$PID/reset>
+
+Reset all internal counters. This is not done atomically and does
+not release any memory.
+
+=item POST http://0/$PID/trim
+
+C<curl --unix-socket $DIR/$PID.sock -XPOST http://0/$PID/trim>
+
+Runs L<malloc_trim(3)> with a 0 pad value to release unused memory
+back to the kernel. In our malloc implementation, this is done
+lazily to avoid contention and does not happen unless sleeping threads.
+
+=item POST http://0/$PID/ctl
+
+Set various internal knobs. Currently, C<X-Mwrap-BT> is the
+only knob supported:
+
+C<curl --unix-socket $DIR/$PID.sock -XPOST -HX-Mwrap-BT:1 http://0/$PID/ctl>
+
+Using the C<X-Mwrap-BT> header allows changing the aforementioned
+C<bt:> value to a specified depth level. As with C<bt:>, only make small
+adjustments as the memory cost can increase exponentially with each step.
+
+It is typically a good idea to reset (C<http://0/$PID/reset>) after changing
+the depth on a running process.
+
+Headers other than C<X-Mwrap-BT> may be accepted in the future to
+tweak other settings.
+
+=back
+
+=head1 CONTACT
+
+Feedback welcome via plain-text mail to L<mailto:mwrap-public@80x24.org>
+
+Mail archives are hosted at L<https://80x24.org/mwrap-public/>
+
+=head1 COPYRIGHT
+
+Copyright all contributors L<mailto:mwrap-public@80x24.org>
+
+License: GPL-3.0+ L<https://www.gnu.org/licenses/gpl-3.0.txt>
+
+Source code is at L<https://80x24.org/mwrap.git/>
+
+=head1 SEE ALSO
+
+L<mwrap-rproxy(1)>, L<Devel::Mwrap(3pm)>, L<https://80x24.org/mwrap-perl.git>
+
+=cut
diff --git a/MANIFEST b/MANIFEST
index 4c5be8b..8d4cdd6 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -2,6 +2,9 @@
.gitignore
.olddoc.yml
COPYING
+Documentation/.gitignore
+Documentation/GNUmakefile
+Documentation/mwrap.pod
MANIFEST
README
Rakefile
@@ -24,4 +27,3 @@ mwrap.gemspec
t/httpd.t
t/test_common.perl
test/test_mwrap.rb
-lib/mwrap/version.rb
diff --git a/README b/README
index 382c5a0..761f87e 100644
--- a/README
+++ b/README
@@ -8,8 +8,8 @@ mwrap wraps all malloc-family calls to trace the Ruby source
location of such calls and bytes allocated at each callsite.
As of mwrap 2.0.0, it can also function as a leak detector
and show live allocations at every call site. Depending on
-your application and workload, the overhead is roughly a 50%
-increase memory and runtime.
+your application and workload, the overhead is roughly a 50-100%
+increase memory and runtime with default settings.
It works best for allocations under GVL, but tries to track
numeric caller addresses for allocations made without GVL so you
@@ -22,15 +22,13 @@ Userspace RCU project: https://liburcu.org/
It does not require recompiling or rebuilding Ruby, but only
supports Ruby 2.7.0 or later on a few platforms:
-* GNU/Linux
-* FreeBSD
+* GNU/Linux (only tested --without-jemalloc, mwrap 3.x provides its own)
-It may work on NetBSD, OpenBSD and DragonFly BSD.
+It may work on FreeBSD, NetBSD, OpenBSD and DragonFly BSD if given
+appropriate build options.
== Install
- # FreeBSD: pkg install liburcu
-
# Debian-based systems: apt-get liburcu-dev
# Install mwrap via RubyGems.org
@@ -64,24 +62,33 @@ or an address retrieved by backtrace_symbols(3). It is
recommended to use the sort(1) command on either of the
first two columns to find the hottest malloc locations.
-mwrap 2.0.0+ also supports a Rack application endpoint,
+mwrap 3.0.0+ also supports an embedded HTTP server
it is documented at:
-https://80x24.org/mwrap/MwrapRack.html
+https://80x24.org/mwrap.git/tree/Documentation/mwrap.pod
== Known problems
* 32-bit machines are prone to overflow (WONTFIX)
-== Public mail archives and contact info:
+* signalfd(2)-reliant code will need latest URCU with commit
+ ea3a28a3f71dd02f (Disable signals in URCU background threads, 2022-09-23)
+ This doesn't affect C Ruby itself, and signalfd(2) use is rare
+ 3rd-party processes.
+
+* Ruby source files over 16.7 million lines long are not supported :P
+
+== Public mail archives (HTTP, Atom feeds, IMAP mailbox, NNTP group, POP3):
* https://80x24.org/mwrap-public/
* nntps://80x24.org/inbox.comp.lang.ruby.mwrap
* imaps://;AUTH=ANONYMOUS@80x24.org/inbox.comp.lang.ruby.mwrap.0
* https://80x24.org/mwrap-public/_/text/help/#pop3
-No subscription will ever be required to post, but HTML mail
-will be rejected:
+No subscription nor real identities will ever be required to obtain support,
+but HTML mail is rejected. Memory usage reductions start with you;
+only send plain-text mail to us and do not top-post. HTML mail and
+top-posting costs everybody memory and bandwidth.
mwrap-public@80x24.org
@@ -89,9 +96,10 @@ will be rejected:
git clone https://80x24.org/mwrap.git
-Send all patches and pull requests (use "git request-pull" to format) to
-mwrap-public@80x24.org. We do not use centralized or proprietary messaging
-systems.
+Send all patches ("git format-patch" + "git send-email") and
+pull requests (use "git request-pull" to format) via email
+to mwrap-perl@80x24.org. We do not and will not use
+proprietary messaging systems.
== License
diff --git a/mwrap.gemspec b/mwrap.gemspec
index dc99924..b6f9e71 100644
--- a/mwrap.gemspec
+++ b/mwrap.gemspec
@@ -1,6 +1,5 @@
git_manifest = `git ls-files 2>/dev/null`.split("\n")
git_ok = $?.success?
-git_manifest << 'lib/mwrap/version.rb'.freeze # generated by ./VERSION-GEN
manifest = File.exist?('MANIFEST') ?
File.readlines('MANIFEST').map!(&:chomp).delete_if(&:empty?) : git_manifest
if git_ok && manifest != git_manifest
@@ -12,11 +11,19 @@ end
version = `./VERSION-GEN`.chomp
$?.success? or abort './VERSION-GEN failed'
+manifest << 'lib/mwrap/version.rb'.freeze
+
+if system(*%w(make -C Documentation man)) ||
+ system(*%w(gmake -C Documentation man))
+ manifest.concat(%w(Documentation/mwrap.1))
+else
+ warn 'failed to build man-page(s), proceeding without them'
+end
Gem::Specification.new do |s|
s.name = 'mwrap'
s.version = version
- s.homepage = 'https://80x24.org/mwrap/'
+ s.homepage = 'https://80x24.org/mwrap.git/'
s.authors = ["mwrap hackers"]
s.summary = 'LD_PRELOAD malloc wrapper for Ruby'
s.executables = %w(mwrap)
^ permalink raw reply related [relevance 2%]
* [ANNOUNCE] mwrap 2.2.0
@ 2022-08-22 18:21 7% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2022-08-22 18:21 UTC (permalink / raw)
To: mwrap-public; +Cc: Sam Saffron
This release adds Ruby 2.7, 3.0, and 3.1 support.
This also fixes breakage in urcu v0.11.4, v0.12.3, and v0.13.1.
While urcu v0.12.4 and v0.13.2 were released 2022-08-18,
old versions will continue to be in distros for a while
(and urcu v0.11.x won't be updated).
There are also several updates ported from the Perl version.
7 changes since v2.1.0 in 2018:
memalign: perform rcu_read_unlock on ENOMEM
workaround breakage from urcu v0.11.4
constify arg for totals_add_rcu
support Ruby 3.0.x
disable HeapPageBody count test for Ruby 3.1
quiet uninitialized and unused variable warnings
various doc updates
Available via RubyGems and https://80x24.org/mwrap/
(I've Bcc-ed some folks since I'm not sure if their addresses are public)
^ permalink raw reply [relevance 7%]
* [PATCH] various doc updates
@ 2022-08-22 18:09 3% Eric Wong
0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2022-08-22 18:09 UTC (permalink / raw)
To: mwrap-public
Copyright years are unnecessary churn and probably aren't
necessary:
https://www.linuxfoundation.org/blog/copyright-notices-in-open-source-software-projects/
add POP3 and IMAP archive info, and drop the outdated link
to the live demo since I no longer have the resources to run it.
Finally, add rdoc (olddoc) + rsync support
---
.document | 1 +
.gitignore | 3 +++
.olddoc.yml | 4 +++-
README | 12 +++++++-----
Rakefile | 30 +++++++++++++++++++++++++++++-
bin/mwrap | 2 +-
ext/mwrap/extconf.rb | 2 +-
ext/mwrap/mwrap.c | 2 +-
lib/mwrap_rack.rb | 8 +++-----
test/test_mwrap.rb | 2 +-
10 files changed, 50 insertions(+), 16 deletions(-)
diff --git a/.document b/.document
index 4ca33e3..4385cfe 100644
--- a/.document
+++ b/.document
@@ -1,2 +1,3 @@
ext/mwrap/mwrap.c
lib/mwrap_rack.rb
+README
diff --git a/.gitignore b/.gitignore
index aa3606c..f670b62 100644
--- a/.gitignore
+++ b/.gitignore
@@ -4,3 +4,6 @@
/pkg
/*.gem
/doc
+/NEWS
+/NEWS.atom.xml
+/LATEST
diff --git a/.olddoc.yml b/.olddoc.yml
index dac0353..bfecbaf 100644
--- a/.olddoc.yml
+++ b/.olddoc.yml
@@ -5,4 +5,6 @@ rdoc_url: https://80x24.org/mwrap/
ml_url: https://80x24.org/mwrap-public/
public_email: mwrap-public@80x24.org
nntp_url:
- - nntp://news.public-inbox.org/inbox.comp.lang.ruby.mwrap
+- nntps://news.public-inbox.org/inbox.comp.lang.ruby.mwrap
+imap_url:
+- imaps://;AUTH=ANONYMOUS@80x24.org/inbox.comp.lang.ruby.mwrap.0
diff --git a/README b/README
index 3a20258..f387bc4 100644
--- a/README
+++ b/README
@@ -67,16 +67,18 @@ first two columns to find the hottest malloc locations.
mwrap 2.0.0+ also supports a Rack application endpoint,
it is documented at:
- https://80x24.org/mwrap/MwrapRack.html
+https://80x24.org/mwrap/MwrapRack.html
== Known problems
* 32-bit machines are prone to overflow (WONTFIX)
-== Mail archives and list:
+== Public mail archives and contact info:
- https://80x24.org/mwrap-public/
- nntp://80x24.org/inbox.comp.lang.ruby.mwrap
+* https://80x24.org/mwrap-public/
+* nntps://80x24.org/inbox.comp.lang.ruby.mwrap
+* imaps://;AUTH=ANONYMOUS@80x24.org/inbox.comp.lang.ruby.mwrap.0
+* https://80x24.org/mwrap-public/_/text/help/#pop3
No subscription will ever be required to post, but HTML mail
will be rejected:
@@ -88,7 +90,7 @@ will be rejected:
git clone https://80x24.org/mwrap.git
Send all patches and pull requests (use "git request-pull" to format) to
-the mailing list. We do not use centralized or proprietary messaging
+mwrap-public@80x24.org. We do not use centralized or proprietary messaging
systems.
== License
diff --git a/Rakefile b/Rakefile
index 50bfa89..255d346 100644
--- a/Rakefile
+++ b/Rakefile
@@ -1,4 +1,4 @@
-# Copyright (C) 2018 mwrap hackers <mwrap-public@80x24.org>
+# Copyright (C) mwrap hackers <mwrap-public@80x24.org>
# License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
require 'rake/testtask'
begin
@@ -14,3 +14,31 @@ task :default => :compile
c_files = File.readlines('MANIFEST').grep(%r{ext/.*\.[ch]$}).map!(&:chomp!)
task 'compile:mwrap' => c_files
+
+olddoc = ENV['OLDDOC'] || 'olddoc'
+rdoc = ENV['RDOC'] || 'rdoc'
+task :rsync_docs do
+ require 'fileutils'
+ top = %w(README COPYING LATEST NEWS NEWS.atom.xml)
+ system("git", "set-file-times")
+ dest = ENV["RSYNC_DEST"] || "80x24.org:/srv/80x24/mwrap/"
+ FileUtils.rm_rf('doc')
+ sh "#{olddoc} prepare"
+ sh "#{rdoc} -f dark216" # dark216 requires olddoc 1.7+
+ File.unlink('doc/created.rid') rescue nil
+ File.unlink('doc/index.html') rescue nil
+ FileUtils.cp(top, 'doc')
+ sh "#{olddoc} merge"
+
+ Dir['doc/**/*'].each do |txt|
+ st = File.stat(txt)
+ if st.file?
+ gz = "#{txt}.gz"
+ tmp = "#{gz}.#$$"
+ sh("gzip --rsyncable -9 <#{txt} >#{tmp}")
+ File.utime(st.atime, st.mtime, tmp) # make nginx gzip_static happy
+ File.rename(tmp, gz)
+ end
+ end
+ sh("rsync --chmod=Fugo=r #{ENV['RSYNC_OPT']} -av doc/ #{dest}/")
+end
diff --git a/bin/mwrap b/bin/mwrap
index 9f67dab..212078c 100755
--- a/bin/mwrap
+++ b/bin/mwrap
@@ -1,6 +1,6 @@
#!/usr/bin/ruby
# frozen_string_literal: true
-# Copyright (C) 2018 mwrap hackers <mwrap-public@80x24.org>
+# Copyright (C) mwrap hackers <mwrap-public@80x24.org>
# License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
require 'mwrap'
mwrap_so = $".grep(%r{/mwrap\.so\z})[0] or abort "mwrap.so not loaded"
diff --git a/ext/mwrap/extconf.rb b/ext/mwrap/extconf.rb
index 254a3bb..512ab82 100644
--- a/ext/mwrap/extconf.rb
+++ b/ext/mwrap/extconf.rb
@@ -1,5 +1,5 @@
# frozen_string_literal: true
-# Copyright (C) 2018 mwrap hackers <mwrap-public@80x24.org>
+# Copyright (C) mwrap hackers <mwrap-public@80x24.org>
# License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
require 'mkmf'
diff --git a/ext/mwrap/mwrap.c b/ext/mwrap/mwrap.c
index a097b2e..85b847b 100644
--- a/ext/mwrap/mwrap.c
+++ b/ext/mwrap/mwrap.c
@@ -1,5 +1,5 @@
/*
- * Copyright (C) 2018 mwrap hackers <mwrap-public@80x24.org>
+ * Copyright (C) mwrap hackers <mwrap-public@80x24.org>
* License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
*/
#define _LGPL_SOURCE /* allows URCU to inline some stuff */
diff --git a/lib/mwrap_rack.rb b/lib/mwrap_rack.rb
index e45b26d..53380b9 100644
--- a/lib/mwrap_rack.rb
+++ b/lib/mwrap_rack.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2018 all contributors <mwrap@80x24.org>
+# Copyright (C) all contributors <mwrap-public@80x24.org>
# License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
# frozen_string_literal: true
require 'mwrap'
@@ -17,9 +17,6 @@ require 'cgi'
# map('/MWRAP') { run(MwrapRack.new) }
# map('/') { run(your_normal_app) }
#
-# A live demo is available at https://80x24.org/MWRAP/
-# (warning the demo machine is 32-bit, so counters will overflow)
-#
# This module is only available in mwrap 2.0.0+
class MwrapRack
module HtmlResponse # :nodoc:
@@ -115,10 +112,11 @@ class MwrapRack
end
GC_STAT_URL = 'https://docs.ruby-lang.org/en/trunk/GC.html#method-c-stat'
- GC_STAT_HELP = <<~""
+ GC_STAT_HELP = <<~EOM
<p>Non-Infinity lifespans can indicate fragmentation.
<p>See <a
href="#{GC_STAT_URL}">#{GC_STAT_URL}</a> for info on GC.stat values.
+ EOM
def each
Mwrap.quiet do
diff --git a/test/test_mwrap.rb b/test/test_mwrap.rb
index c506554..eaa65cb 100644
--- a/test/test_mwrap.rb
+++ b/test/test_mwrap.rb
@@ -1,5 +1,5 @@
# frozen_string_literal: true
-# Copyright (C) 2018 mwrap hackers <mwrap-public@80x24.org>
+# Copyright (C) mwrap hackers <mwrap-public@80x24.org>
# License: GPL-2.0+ <https://www.gnu.org/licenses/gpl-2.0.txt>
require 'test/unit'
require 'mwrap'
^ permalink raw reply related [relevance 3%]
Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2022-08-22 18:09 3% [PATCH] various doc updates Eric Wong
2022-08-22 18:21 7% [ANNOUNCE] mwrap 2.2.0 Eric Wong
2023-01-09 5:52 2% [PATCH] various doc updates, add mwrap(1) manpage Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/mwrap.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).