* [PATCH 1/2] buffer/splice: prepare for IO::Splice::WAITALL removal
@ 2015-02-18 22:23 Eric Wong
2015-02-18 22:23 ` [PATCH 2/2] buffer: allow limiting the amount of bytes output Eric Wong
0 siblings, 1 reply; 2+ messages in thread
From: Eric Wong @ 2015-02-18 22:23 UTC (permalink / raw)
To: dtas-all
This feature in the io_splice was probably a bad idea
and slated for removal at some point in the future.
Anyways, do not rely on it since it is undocumented.
---
lib/dtas/buffer/splice.rb | 25 ++++++++++++++++++++++---
1 file changed, 22 insertions(+), 3 deletions(-)
diff --git a/lib/dtas/buffer/splice.rb b/lib/dtas/buffer/splice.rb
index c83b87b..b987f3a 100644
--- a/lib/dtas/buffer/splice.rb
+++ b/lib/dtas/buffer/splice.rb
@@ -12,7 +12,6 @@ module DTAS::Buffer::Splice # :nodoc:
MAX_SIZE = File.read("/proc/sys/fs/pipe-max-size").to_i
DEVNULL = File.open("/dev/null", "r+")
F_MOVE = IO::Splice::F_MOVE
- WAITALL = IO::Splice::WAITALL
def buffer_size
@to_io.pipe_size
@@ -44,6 +43,26 @@ def broadcast_one(targets)
nil # do not return error here, we already spewed an error message
end
+ def __tee_in_full(src, dst, bytes)
+ rv = 0
+ while bytes > 0
+ s = IO.tee(src, dst, bytes)
+ bytes -= s
+ rv += s
+ end
+ rv
+ end
+
+ def __splice_in_full(src, dst, bytes, flags)
+ rv = 0
+ while bytes > 0
+ s = IO.splice(src, nil, dst, nil, bytes, flags)
+ rv += s
+ bytes -= s
+ end
+ rv
+ end
+
# returns the largest value we teed
def __broadcast_tee(blocked, targets, chunk_size)
most_teed = 0
@@ -51,7 +70,7 @@ def __broadcast_tee(blocked, targets, chunk_size)
begin
t = (dst.nonblock? || most_teed == 0) ?
IO.trytee(@to_io, dst, chunk_size) :
- IO.tee(@to_io, dst, chunk_size, WAITALL)
+ __tee_in_full(@to_io, dst, chunk_size)
if Integer === t
if t > most_teed
chunk_size = t if most_teed == 0
@@ -117,7 +136,7 @@ def broadcast_inf(targets)
end
else
# the blocking case is simple
- s = IO.splice(@to_io, nil, last, nil, bytes, WAITALL|F_MOVE)
+ s = __splice_in_full(@to_io, last, bytes, F_MOVE)
end
@bytes_xfer += s
--
EW
^ permalink raw reply related [flat|nested] 2+ messages in thread
* [PATCH 2/2] buffer: allow limiting the amount of bytes output
2015-02-18 22:23 [PATCH 1/2] buffer/splice: prepare for IO::Splice::WAITALL removal Eric Wong
@ 2015-02-18 22:23 ` Eric Wong
0 siblings, 0 replies; 2+ messages in thread
From: Eric Wong @ 2015-02-18 22:23 UTC (permalink / raw)
To: dtas-all
This will aid in allowing us to create effects which affect
only a certain part of a track.
---
lib/dtas/buffer.rb | 6 +++---
lib/dtas/buffer/read_write.rb | 9 +++++----
lib/dtas/buffer/splice.rb | 9 +++++----
3 files changed, 13 insertions(+), 11 deletions(-)
diff --git a/lib/dtas/buffer.rb b/lib/dtas/buffer.rb
index b03ed8a..23b0b77 100644
--- a/lib/dtas/buffer.rb
+++ b/lib/dtas/buffer.rb
@@ -48,14 +48,14 @@ def __dst_error(dst, e)
# - subset of targets array for :wait_writable
# - some type of StandardError
# - nil
- def broadcast(targets)
+ def broadcast(targets, limit = nil)
case targets.size
when 0
:ignore # this will pause decoders
when 1
- broadcast_one(targets)
+ broadcast_one(targets, limit)
else # infinity
- broadcast_inf(targets)
+ broadcast_inf(targets, limit)
end
end
diff --git a/lib/dtas/buffer/read_write.rb b/lib/dtas/buffer/read_write.rb
index a27b823..64ad297 100644
--- a/lib/dtas/buffer/read_write.rb
+++ b/lib/dtas/buffer/read_write.rb
@@ -26,9 +26,9 @@ def discard(bytes)
end
# always block when we have a single target
- def broadcast_one(targets)
+ def broadcast_one(targets, limit = nil)
buf = _rbuf
- @to_io.read_nonblock(MAX_AT_ONCE, buf)
+ @to_io.read_nonblock(limit || MAX_AT_ONCE, buf)
n = targets[0].write(buf) # IO#write has write-in-full behavior
@bytes_xfer += n
:wait_readable
@@ -42,7 +42,7 @@ def broadcast_one(targets)
nil # do not return error here, we already spewed an error message
end
- def broadcast_inf(targets)
+ def broadcast_inf(targets, limit = nil)
nr_nb = targets.count(&:nonblock?)
if nr_nb == 0 || nr_nb == targets.size
# if all targets are full, don't start until they're all writable
@@ -61,7 +61,8 @@ def broadcast_inf(targets)
# don't pin too much on one target
bytes = inflight
- bytes = bytes > MAX_AT_ONCE ? MAX_AT_ONCE : bytes
+ limit ||= MAX_AT_ONCE
+ bytes = bytes > limit ? limit : bytes
buf = _rbuf
@to_io.read(bytes, buf)
n = buf.bytesize
diff --git a/lib/dtas/buffer/splice.rb b/lib/dtas/buffer/splice.rb
index b987f3a..02ce877 100644
--- a/lib/dtas/buffer/splice.rb
+++ b/lib/dtas/buffer/splice.rb
@@ -28,9 +28,10 @@ def discard(bytes)
IO.splice(@to_io, nil, DEVNULL, nil, bytes)
end
- def broadcast_one(targets)
+ def broadcast_one(targets, limit = nil)
# single output is always non-blocking
- s = IO.trysplice(@to_io, nil, targets[0], nil, MAX_AT_ONCE_1, F_MOVE)
+ limit ||= MAX_AT_ONCE_1
+ s = IO.trysplice(@to_io, nil, targets[0], nil, limit, F_MOVE)
if Symbol === s
targets # our one and only target blocked on write
else
@@ -88,7 +89,7 @@ def __broadcast_tee(blocked, targets, chunk_size)
most_teed
end
- def broadcast_inf(targets)
+ def broadcast_inf(targets, limit = nil)
if targets.all?(&:ready_write_optimized?)
blocked = []
elsif targets.none?(&:nonblock?)
@@ -105,7 +106,7 @@ def broadcast_inf(targets)
end
# don't pin too much on one target
- bytes = MAX_AT_ONCE
+ bytes = limit || MAX_AT_ONCE
last = targets.pop # we splice to the last one, tee to the rest
# this may return zero if all targets were non-blocking
--
EW
^ permalink raw reply related [flat|nested] 2+ messages in thread
end of thread, other threads:[~2015-02-18 22:53 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-02-18 22:23 [PATCH 1/2] buffer/splice: prepare for IO::Splice::WAITALL removal Eric Wong
2015-02-18 22:23 ` [PATCH 2/2] buffer: allow limiting the amount of bytes output Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/dtas.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).