* [PATCH] use sleepy_penguin 3.5+ for splice and tee support
@ 2019-01-02 20:42 5% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-01-02 20:42 UTC (permalink / raw)
To: dtas-all
Eliminate loading of the io_splice RubyGem to reduce memory overhead.
Extra DSOs are wasteful and io_splice is being phased oiut for
sleepy_penguin, which encapsulates more Linux-specific functionality
anyways.
cf. https://udrepper.livejournal.com/8790.html
---
INSTALL | 8 +++-----
dtas-linux.gemspec | 5 ++---
lib/dtas/buffer.rb | 4 +++-
lib/dtas/buffer/read_write.rb | 2 +-
lib/dtas/buffer/splice.rb | 20 +++++++++++---------
lib/dtas/pipe.rb | 13 ++++++++-----
test/test_buffer.rb | 22 +++++++++++++---------
test/test_sink_pipe_size.rb | 4 ++--
8 files changed, 43 insertions(+), 35 deletions(-)
diff --git a/INSTALL b/INSTALL
index 96cfcdb..b07b3c7 100644
--- a/INSTALL
+++ b/INSTALL
@@ -19,13 +19,13 @@ Debian 7+ users can install dependencies easily:
# installing dtas RubyGem on GNU/Linux (Linux kernel 2.6.32+)
Be sure to have Ruby development headers and a working C compiler.
-This will pull in the io_splice and sleepy_penguin RubyGems for minor
+This will pull in the sleepy_penguin RubyGems for minor
speedups. If you cannot be bothered to have a development
environment, just use "gem install dtas".
sudo gem install dtas-linux
-This should pull in the "io_splice" and "sleepy_penguin" RubyGems
+This should pull in the "sleepy_penguin" RubyGems
For future upgrades of dtas (upgrades to dtas-linux will be infrequent)
@@ -45,10 +45,8 @@ Grab the latest tarball from our HTTPS site:
$ cd dtas-0.15.0
$ sudo ruby setup.rb
-GNU/Linux users may optionally install "io_splice" and
-"sleepy_penguin" packages:
+GNU/Linux users may optionally install the "sleepy_penguin" package:
- * io_splice - https://bogomips.org/ruby_io_splice/
* sleepy_penguin - https://bogomips.org/sleepy_penguin/
# CONTACT
diff --git a/dtas-linux.gemspec b/dtas-linux.gemspec
index ca1be89..0c1d97e 100644
--- a/dtas-linux.gemspec
+++ b/dtas-linux.gemspec
@@ -13,8 +13,7 @@
s.email = %q{e@80x24.org}
s.files = []
s.homepage = 'https://80x24.org/dtas/'
- s.add_dependency(%q<dtas>)
- s.add_dependency(%q<io_splice>, '~> 4')
- s.add_dependency(%q<sleepy_penguin>, '~> 3')
+ s.add_dependency(%q<dtas>, '~> 0.16')
+ s.add_dependency(%q<sleepy_penguin>, '~> 3.5')
s.licenses = 'GPL-3.0+'
end
diff --git a/lib/dtas/buffer.rb b/lib/dtas/buffer.rb
index 6f43dfa..c3d8ee2 100644
--- a/lib/dtas/buffer.rb
+++ b/lib/dtas/buffer.rb
@@ -8,7 +8,9 @@
class DTAS::Buffer # :nodoc:
begin
raise LoadError, "no splice with _DTAS_POSIX" if ENV["_DTAS_POSIX"]
- require 'io/splice' # splice is only in Linux for now...
+ require 'sleepy_penguin' # splice is only in Linux for now...
+ SleepyPenguin.respond_to?(:splice) or
+ raise LoadError, 'sleepy_penguin 3.5+ required for splice', []
require_relative 'buffer/splice'
include DTAS::Buffer::Splice
rescue LoadError
diff --git a/lib/dtas/buffer/read_write.rb b/lib/dtas/buffer/read_write.rb
index 5845309..06947a5 100644
--- a/lib/dtas/buffer/read_write.rb
+++ b/lib/dtas/buffer/read_write.rb
@@ -7,7 +7,7 @@
require_relative '../nonblock'
# compatibility code for systems lacking "splice" support via the
-# "io-splice" RubyGem. Used only by -player
+# "sleepy_penguin" 3.5+ RubyGem. Used only by -player
module DTAS::Buffer::ReadWrite # :nodoc:
MAX_AT_ONCE = 512 # min PIPE_BUF value in POSIX
attr_accessor :buffer_size
diff --git a/lib/dtas/buffer/splice.rb b/lib/dtas/buffer/splice.rb
index 1cb0fbc..cd00bbb 100644
--- a/lib/dtas/buffer/splice.rb
+++ b/lib/dtas/buffer/splice.rb
@@ -2,15 +2,17 @@
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'io/nonblock'
-require 'io/splice'
+require 'sleepy_penguin'
require_relative '../../dtas'
require_relative '../pipe'
-# Used by -player on Linux systems with the "io-splice" RubyGem installed
+# Used by -player on Linux systems with the "sleepy_penguin" RubyGem installed
module DTAS::Buffer::Splice # :nodoc:
MAX_AT_ONCE = 4096 # page size in Linux
MAX_AT_ONCE_1 = 65536
- F_MOVE = IO::Splice::F_MOVE
+ F_MOVE = SleepyPenguin::F_MOVE
+ F_NONBLOCK = SleepyPenguin::F_NONBLOCK
+ TRY = { exception: false }.freeze
def buffer_size
@to_io.pipe_size
@@ -24,13 +26,13 @@ def buffer_size=(bytes)
# be sure to only call this with nil when all writers to @wr are done
def discard(bytes)
- IO.splice(@to_io, nil, DTAS.null, nil, bytes)
+ SleepyPenguin.splice(@to_io, DTAS.null, bytes)
end
def broadcast_one(targets, limit = nil)
# single output is always non-blocking
limit ||= MAX_AT_ONCE_1
- s = IO.trysplice(@to_io, nil, targets[0], nil, limit, F_MOVE)
+ s = SleepyPenguin.splice(@to_io, targets[0], limit, F_MOVE, TRY)
if Symbol === s
targets # our one and only target blocked on write
else
@@ -46,7 +48,7 @@ def broadcast_one(targets, limit = nil)
def __tee_in_full(src, dst, bytes)
rv = 0
while bytes > 0
- s = IO.tee(src, dst, bytes)
+ s = SleepyPenguin.tee(src, dst, bytes)
bytes -= s
rv += s
end
@@ -56,7 +58,7 @@ def __tee_in_full(src, dst, bytes)
def __splice_in_full(src, dst, bytes, flags)
rv = 0
while bytes > 0
- s = IO.splice(src, nil, dst, nil, bytes, flags)
+ s = SleepyPenguin.splice(src, dst, bytes, flags)
rv += s
bytes -= s
end
@@ -69,7 +71,7 @@ def __broadcast_tee(blocked, targets, chunk_size)
targets.delete_if do |dst|
begin
t = (dst.nonblock? || most_teed == 0) ?
- IO.trytee(@to_io, dst, chunk_size) :
+ SleepyPenguin.tee(@to_io, dst, chunk_size, F_NONBLOCK, TRY) :
__tee_in_full(@to_io, dst, chunk_size)
if Integer === t
if t > most_teed
@@ -117,7 +119,7 @@ def broadcast_inf(targets, limit = nil)
begin
targets << last
if last.nonblock? || most_teed == 0
- s = IO.trysplice(@to_io, nil, last, nil, bytes, F_MOVE)
+ s = SleepyPenguin.splice(@to_io, last, bytes, F_MOVE|F_NONBLOCK, TRY)
if Symbol === s
blocked << last
diff --git a/lib/dtas/pipe.rb b/lib/dtas/pipe.rb
index f9d5149..97cbd71 100644
--- a/lib/dtas/pipe.rb
+++ b/lib/dtas/pipe.rb
@@ -2,7 +2,7 @@
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
begin
- require 'io/splice'
+ require 'sleepy_penguin'
rescue LoadError
end
require_relative '../dtas'
@@ -20,12 +20,15 @@ def self.new
rv
end
- # create no-op methods for non-Linux
- unless method_defined?(:pipe_size=)
- def pipe_size=(_)
- end
+ def pipe_size=(nr)
+ defined?(SleepyPenguin::F_SETPIPE_SZ) and
+ fcntl(SleepyPenguin::F_SETPIPE_SZ, nr)
end
+ def pipe_size
+ fcntl(SleepyPenguin::F_GETPIPE_SZ)
+ end if defined?(SleepyPenguin::F_GETPIPE_SZ)
+
# avoid syscall, we never change IO#nonblock= directly
def nonblock?
false
diff --git a/test/test_buffer.rb b/test/test_buffer.rb
index 0fb4fba..b21a69f 100644
--- a/test/test_buffer.rb
+++ b/test/test_buffer.rb
@@ -49,14 +49,14 @@ def test_set_buffer_size
buf = new_buffer
buf.buffer_size = @@max_size
assert_equal @@max_size, buf.buffer_size
- end if @@max_size
+ end if defined?(SleepyPenguin::F_GETPIPE_SZ)
def test_buffer_size
buf = new_buffer
assert_operator buf.buffer_size, :>, 128
buf.buffer_size = @@max_size
assert_equal @@max_size, buf.buffer_size
- end if @@max_size
+ end if defined?(SleepyPenguin::F_GETPIPE_SZ)
def test_broadcast_1
buf = new_buffer
@@ -87,9 +87,9 @@ def test_broadcast_tee
assert_empty blocked
assert_equal "HELLO", a[0].read(5)
assert_equal "HELLO", b[0].read(5)
- max = '*' * a[0].pipe_size
+ max = '*' * pipe_size(a[0])
assert_equal max.size, a[1].write(max)
- assert_equal a[0].nread, a[0].pipe_size
+ assert_equal a[0].nread, pipe_size(a[0])
a[1].nonblock = true
assert_equal 5, buf.__broadcast_tee(blocked, [a[1], b[1]], 5)
assert_equal [a[1]], blocked
@@ -108,10 +108,10 @@ def test_broadcast
assert_equal "HELLO", a[0].read(5)
assert_equal "HELLO", b[0].read(5)
- return unless b[1].respond_to?(:pipe_size)
+ return unless defined?(SleepyPenguin::F_GETPIPE_SZ)
b[1].nonblock = true
- b[1].write('*' * b[1].pipe_size)
+ b[1].write('*' * pipe_size(b[1]))
buf.wr.write "BYE"
assert_equal :wait_readable, buf.broadcast([a[1], b[1]])
assert_equal 8, buf.bytes_xfer
@@ -157,8 +157,8 @@ def test_broadcast_all_full
a = pipe
b = pipe
buf = new_buffer
- a[1].write('*' * a[1].pipe_size)
- b[1].write('*' * b[1].pipe_size)
+ a[1].write('*' * pipe_size(a[1]))
+ b[1].write('*' * pipe_size(b[1]))
a[1].nonblock = true
b[1].nonblock = true
@@ -167,7 +167,7 @@ def test_broadcast_all_full
buf.wr.write "HELLO"
assert_equal tmp, buf.broadcast(tmp)
assert_equal [a[1], b[1]], tmp
- end if IO.method_defined?(:pipe_size)
+ end if defined?(SleepyPenguin::F_GETPIPE_SZ)
def test_serialize
buf = new_buffer
@@ -204,4 +204,8 @@ def test_load_size
assert_equal 4096, buf.buffer_size
buf.close!
end
+
+ def pipe_size(io)
+ io.fcntl(SleepyPenguin::F_GETPIPE_SZ)
+ end
end
diff --git a/test/test_sink_pipe_size.rb b/test/test_sink_pipe_size.rb
index 645084a..1ca3108 100644
--- a/test/test_sink_pipe_size.rb
+++ b/test/test_sink_pipe_size.rb
@@ -2,7 +2,7 @@
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
begin
- require 'io/splice'
+ require 'sleepy_penguin'
require './test/player_integration'
class TestSinkPipeSizeIntegration < Testcase
include PlayerIntegration
@@ -14,7 +14,7 @@ def test_sink_pipe_size_integration
s.req_ok("sink ed default pipe_size=0x10000")
s.req_ok("sink ed default pipe_size=")
s.req_ok("sink ed default pipe_size=4096")
- end if IO.method_defined?(:pipe_size=)
+ end if SleepyPenguin.const_defined?(:F_SETPIPE_SZ)
end
rescue LoadError
end
--
EW
^ permalink raw reply related [relevance 5%]
* [ANN] dtas 0.16.0 - duct tape audio suite for *nix
@ 2019-01-02 21:35 7% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2019-01-02 21:35 UTC (permalink / raw)
To: ruby-talk, dtas-all
Free Software command-line tools for audio playback, mastering, and
whatever else related to audio. dtas follows the worse-is-better
philosophy and acts as duct tape to combine existing command-line tools
for flexibility and ease-of-development. dtas is currently implemented
in Ruby (and some embedded shell), but may use other languages in the
future.
Changes:
A bunch of minor fixes and cleanups accumulating for the past
two years since the last release. It's tough to remember to
make releases when I'm always running the latest version from
git :x
Most notably, "io_splice" is no longer used for dtas-linux
users since "sleepy_penguin" includes all the functionality
we use. This is to reduce memory overhead from extra DSOs(*)
There's also some deprecation warning fixes for the
still-undocumented "dtas-mlib" command.
12 changes since v0.15.0 (2017-04-07):
pipeline: new module for running process pipelines
console: ensure time calculations are done in UTC
Rakefile: update path for uploads
player: support guessing encodings for comments
get rid of Windows-31J regexps
mlib: compatibility with Sequel 5.x
mlib: remove redundant tag massaging and encoding
mlib: use flock to get around SQLite busy errors
mlib: ignore files with nil times
dtas/watchable: check SystemCallError
mlib: fix unused variable warning
use sleepy_penguin 3.5+ for splice and tee support
(*) https://udrepper.livejournal.com/8790.html
* homepage: https://80x24.org/dtas/README
* https://80x24.org/dtas/INSTALL
* https://80x24.org/dtas/dtas-player.txt
* https://80x24.org/dtas/NEWS.atom
* git clone https://80x24.org/dtas.git
* dtas-all@nongnu.org (plain-text only, no HTML mail, please)
* mailing list archives: https://80x24.org/dtas-all/
nntp://news.public-inbox.org/inbox.comp.audio.dtas
https://80x24.org/dtas-all/new.atom
^ permalink raw reply [relevance 7%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-01-02 21:35 7% [ANN] dtas 0.16.0 - duct tape audio suite for *nix Eric Wong
2019-01-02 20:42 5% [PATCH] use sleepy_penguin 3.5+ for splice and tee support Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/dtas.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).