From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on dcvr.yhbt.net X-Spam-Level: X-Spam-ASN: AS22989 208.118.235.0/24 X-Spam-Status: No, score=-4.2 required=3.0 tests=AWL,BAYES_00, HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RCVD_IN_DNSWL_HI, SPF_PASS shortcircuit=no autolearn=ham autolearn_force=no version=3.4.2 Received: from lists.gnu.org (lists.gnu.org [208.118.235.17]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by dcvr.yhbt.net (Postfix) with ESMTPS id 69D321F770 for ; Wed, 2 Jan 2019 20:42:40 +0000 (UTC) Received: from localhost ([127.0.0.1]:47287 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1genLa-0001nN-Fp for e@80x24.org; Wed, 02 Jan 2019 15:42:38 -0500 Received: from eggs.gnu.org ([208.118.235.92]:45937) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1genLV-0001m7-BC for dtas-all@nongnu.org; Wed, 02 Jan 2019 15:42:36 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1genLR-0005xX-Sf for dtas-all@nongnu.org; Wed, 02 Jan 2019 15:42:33 -0500 Received: from dcvr.yhbt.net ([64.71.152.64]:35346) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1genLR-0005lB-FM for dtas-all@nongnu.org; Wed, 02 Jan 2019 15:42:29 -0500 Received: from localhost (dcvr.yhbt.net [127.0.0.1]) by dcvr.yhbt.net (Postfix) with ESMTP id E6FB81F770 for ; Wed, 2 Jan 2019 20:42:12 +0000 (UTC) From: Eric Wong To: dtas-all@nongnu.org Subject: [PATCH] use sleepy_penguin 3.5+ for splice and tee support Date: Wed, 2 Jan 2019 20:42:12 +0000 Message-Id: <20190102204212.18266-1-e@80x24.org> MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-Received-From: 64.71.152.64 X-BeenThere: dtas-all@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: duct tape audio suite List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: dtas-all-bounces+e=80x24.org@nongnu.org Sender: "dtas-all" Eliminate loading of the io_splice RubyGem to reduce memory overhead. Extra DSOs are wasteful and io_splice is being phased oiut for sleepy_penguin, which encapsulates more Linux-specific functionality anyways. cf. https://udrepper.livejournal.com/8790.html --- INSTALL | 8 +++----- dtas-linux.gemspec | 5 ++--- lib/dtas/buffer.rb | 4 +++- lib/dtas/buffer/read_write.rb | 2 +- lib/dtas/buffer/splice.rb | 20 +++++++++++--------- lib/dtas/pipe.rb | 13 ++++++++----- test/test_buffer.rb | 22 +++++++++++++--------- test/test_sink_pipe_size.rb | 4 ++-- 8 files changed, 43 insertions(+), 35 deletions(-) diff --git a/INSTALL b/INSTALL index 96cfcdb..b07b3c7 100644 --- a/INSTALL +++ b/INSTALL @@ -19,13 +19,13 @@ Debian 7+ users can install dependencies easily: # installing dtas RubyGem on GNU/Linux (Linux kernel 2.6.32+) =20 Be sure to have Ruby development headers and a working C compiler. -This will pull in the io_splice and sleepy_penguin RubyGems for minor +This will pull in the sleepy_penguin RubyGems for minor speedups. If you cannot be bothered to have a development environment, just use "gem install dtas". =20 sudo gem install dtas-linux =20 -This should pull in the "io_splice" and "sleepy_penguin" RubyGems +This should pull in the "sleepy_penguin" RubyGems =20 For future upgrades of dtas (upgrades to dtas-linux will be infrequent) =20 @@ -45,10 +45,8 @@ Grab the latest tarball from our HTTPS site: $ cd dtas-0.15.0 $ sudo ruby setup.rb =20 -GNU/Linux users may optionally install "io_splice" and -"sleepy_penguin" packages: +GNU/Linux users may optionally install the "sleepy_penguin" package: =20 - * io_splice - https://bogomips.org/ruby_io_splice/ * sleepy_penguin - https://bogomips.org/sleepy_penguin/ =20 # CONTACT diff --git a/dtas-linux.gemspec b/dtas-linux.gemspec index ca1be89..0c1d97e 100644 --- a/dtas-linux.gemspec +++ b/dtas-linux.gemspec @@ -13,8 +13,7 @@ s.email =3D %q{e@80x24.org} s.files =3D [] s.homepage =3D 'https://80x24.org/dtas/' - s.add_dependency(%q) - s.add_dependency(%q, '~> 4') - s.add_dependency(%q, '~> 3') + s.add_dependency(%q, '~> 0.16') + s.add_dependency(%q, '~> 3.5') s.licenses =3D 'GPL-3.0+' end diff --git a/lib/dtas/buffer.rb b/lib/dtas/buffer.rb index 6f43dfa..c3d8ee2 100644 --- a/lib/dtas/buffer.rb +++ b/lib/dtas/buffer.rb @@ -8,7 +8,9 @@ class DTAS::Buffer # :nodoc: begin raise LoadError, "no splice with _DTAS_POSIX" if ENV["_DTAS_POSIX"] - require 'io/splice' # splice is only in Linux for now... + require 'sleepy_penguin' # splice is only in Linux for now... + SleepyPenguin.respond_to?(:splice) or + raise LoadError, 'sleepy_penguin 3.5+ required for splice', [] require_relative 'buffer/splice' include DTAS::Buffer::Splice rescue LoadError diff --git a/lib/dtas/buffer/read_write.rb b/lib/dtas/buffer/read_write.r= b index 5845309..06947a5 100644 --- a/lib/dtas/buffer/read_write.rb +++ b/lib/dtas/buffer/read_write.rb @@ -7,7 +7,7 @@ require_relative '../nonblock' =20 # compatibility code for systems lacking "splice" support via the -# "io-splice" RubyGem. Used only by -player +# "sleepy_penguin" 3.5+ RubyGem. Used only by -player module DTAS::Buffer::ReadWrite # :nodoc: MAX_AT_ONCE =3D 512 # min PIPE_BUF value in POSIX attr_accessor :buffer_size diff --git a/lib/dtas/buffer/splice.rb b/lib/dtas/buffer/splice.rb index 1cb0fbc..cd00bbb 100644 --- a/lib/dtas/buffer/splice.rb +++ b/lib/dtas/buffer/splice.rb @@ -2,15 +2,17 @@ # License: GPL-3.0+ # frozen_string_literal: true require 'io/nonblock' -require 'io/splice' +require 'sleepy_penguin' require_relative '../../dtas' require_relative '../pipe' =20 -# Used by -player on Linux systems with the "io-splice" RubyGem installe= d +# Used by -player on Linux systems with the "sleepy_penguin" RubyGem ins= talled module DTAS::Buffer::Splice # :nodoc: MAX_AT_ONCE =3D 4096 # page size in Linux MAX_AT_ONCE_1 =3D 65536 - F_MOVE =3D IO::Splice::F_MOVE + F_MOVE =3D SleepyPenguin::F_MOVE + F_NONBLOCK =3D SleepyPenguin::F_NONBLOCK + TRY =3D { exception: false }.freeze =20 def buffer_size @to_io.pipe_size @@ -24,13 +26,13 @@ def buffer_size=3D(bytes) =20 # be sure to only call this with nil when all writers to @wr are done def discard(bytes) - IO.splice(@to_io, nil, DTAS.null, nil, bytes) + SleepyPenguin.splice(@to_io, DTAS.null, bytes) end =20 def broadcast_one(targets, limit =3D nil) # single output is always non-blocking limit ||=3D MAX_AT_ONCE_1 - s =3D IO.trysplice(@to_io, nil, targets[0], nil, limit, F_MOVE) + s =3D SleepyPenguin.splice(@to_io, targets[0], limit, F_MOVE, TRY) if Symbol =3D=3D=3D s targets # our one and only target blocked on write else @@ -46,7 +48,7 @@ def broadcast_one(targets, limit =3D nil) def __tee_in_full(src, dst, bytes) rv =3D 0 while bytes > 0 - s =3D IO.tee(src, dst, bytes) + s =3D SleepyPenguin.tee(src, dst, bytes) bytes -=3D s rv +=3D s end @@ -56,7 +58,7 @@ def __tee_in_full(src, dst, bytes) def __splice_in_full(src, dst, bytes, flags) rv =3D 0 while bytes > 0 - s =3D IO.splice(src, nil, dst, nil, bytes, flags) + s =3D SleepyPenguin.splice(src, dst, bytes, flags) rv +=3D s bytes -=3D s end @@ -69,7 +71,7 @@ def __broadcast_tee(blocked, targets, chunk_size) targets.delete_if do |dst| begin t =3D (dst.nonblock? || most_teed =3D=3D 0) ? - IO.trytee(@to_io, dst, chunk_size) : + SleepyPenguin.tee(@to_io, dst, chunk_size, F_NONBLOCK, TRY) = : __tee_in_full(@to_io, dst, chunk_size) if Integer =3D=3D=3D t if t > most_teed @@ -117,7 +119,7 @@ def broadcast_inf(targets, limit =3D nil) begin targets << last if last.nonblock? || most_teed =3D=3D 0 - s =3D IO.trysplice(@to_io, nil, last, nil, bytes, F_MOVE) + s =3D SleepyPenguin.splice(@to_io, last, bytes, F_MOVE|F_NONBLOC= K, TRY) if Symbol =3D=3D=3D s blocked << last =20 diff --git a/lib/dtas/pipe.rb b/lib/dtas/pipe.rb index f9d5149..97cbd71 100644 --- a/lib/dtas/pipe.rb +++ b/lib/dtas/pipe.rb @@ -2,7 +2,7 @@ # License: GPL-3.0+ # frozen_string_literal: true begin - require 'io/splice' + require 'sleepy_penguin' rescue LoadError end require_relative '../dtas' @@ -20,12 +20,15 @@ def self.new rv end =20 - # create no-op methods for non-Linux - unless method_defined?(:pipe_size=3D) - def pipe_size=3D(_) - end + def pipe_size=3D(nr) + defined?(SleepyPenguin::F_SETPIPE_SZ) and + fcntl(SleepyPenguin::F_SETPIPE_SZ, nr) end =20 + def pipe_size + fcntl(SleepyPenguin::F_GETPIPE_SZ) + end if defined?(SleepyPenguin::F_GETPIPE_SZ) + # avoid syscall, we never change IO#nonblock=3D directly def nonblock? false diff --git a/test/test_buffer.rb b/test/test_buffer.rb index 0fb4fba..b21a69f 100644 --- a/test/test_buffer.rb +++ b/test/test_buffer.rb @@ -49,14 +49,14 @@ def test_set_buffer_size buf =3D new_buffer buf.buffer_size =3D @@max_size assert_equal @@max_size, buf.buffer_size - end if @@max_size + end if defined?(SleepyPenguin::F_GETPIPE_SZ) =20 def test_buffer_size buf =3D new_buffer assert_operator buf.buffer_size, :>, 128 buf.buffer_size =3D @@max_size assert_equal @@max_size, buf.buffer_size - end if @@max_size + end if defined?(SleepyPenguin::F_GETPIPE_SZ) =20 def test_broadcast_1 buf =3D new_buffer @@ -87,9 +87,9 @@ def test_broadcast_tee assert_empty blocked assert_equal "HELLO", a[0].read(5) assert_equal "HELLO", b[0].read(5) - max =3D '*' * a[0].pipe_size + max =3D '*' * pipe_size(a[0]) assert_equal max.size, a[1].write(max) - assert_equal a[0].nread, a[0].pipe_size + assert_equal a[0].nread, pipe_size(a[0]) a[1].nonblock =3D true assert_equal 5, buf.__broadcast_tee(blocked, [a[1], b[1]], 5) assert_equal [a[1]], blocked @@ -108,10 +108,10 @@ def test_broadcast assert_equal "HELLO", a[0].read(5) assert_equal "HELLO", b[0].read(5) =20 - return unless b[1].respond_to?(:pipe_size) + return unless defined?(SleepyPenguin::F_GETPIPE_SZ) =20 b[1].nonblock =3D true - b[1].write('*' * b[1].pipe_size) + b[1].write('*' * pipe_size(b[1])) buf.wr.write "BYE" assert_equal :wait_readable, buf.broadcast([a[1], b[1]]) assert_equal 8, buf.bytes_xfer @@ -157,8 +157,8 @@ def test_broadcast_all_full a =3D pipe b =3D pipe buf =3D new_buffer - a[1].write('*' * a[1].pipe_size) - b[1].write('*' * b[1].pipe_size) + a[1].write('*' * pipe_size(a[1])) + b[1].write('*' * pipe_size(b[1])) =20 a[1].nonblock =3D true b[1].nonblock =3D true @@ -167,7 +167,7 @@ def test_broadcast_all_full buf.wr.write "HELLO" assert_equal tmp, buf.broadcast(tmp) assert_equal [a[1], b[1]], tmp - end if IO.method_defined?(:pipe_size) + end if defined?(SleepyPenguin::F_GETPIPE_SZ) =20 def test_serialize buf =3D new_buffer @@ -204,4 +204,8 @@ def test_load_size assert_equal 4096, buf.buffer_size buf.close! end + + def pipe_size(io) + io.fcntl(SleepyPenguin::F_GETPIPE_SZ) + end end diff --git a/test/test_sink_pipe_size.rb b/test/test_sink_pipe_size.rb index 645084a..1ca3108 100644 --- a/test/test_sink_pipe_size.rb +++ b/test/test_sink_pipe_size.rb @@ -2,7 +2,7 @@ # License: GPL-3.0+ # frozen_string_literal: true begin - require 'io/splice' + require 'sleepy_penguin' require './test/player_integration' class TestSinkPipeSizeIntegration < Testcase include PlayerIntegration @@ -14,7 +14,7 @@ def test_sink_pipe_size_integration s.req_ok("sink ed default pipe_size=3D0x10000") s.req_ok("sink ed default pipe_size=3D") s.req_ok("sink ed default pipe_size=3D4096") - end if IO.method_defined?(:pipe_size=3D) + end if SleepyPenguin.const_defined?(:F_SETPIPE_SZ) end rescue LoadError end --=20 EW