* [PATCH 0/5] use fiddle for Linux-specific APIs
@ 2019-12-01 1:26 6% Eric Wong
2019-12-01 1:26 5% ` [PATCH 4/5] watchable: use fiddle for inotify support Eric Wong
0 siblings, 1 reply; 5+ results
From: Eric Wong @ 2019-12-01 1:26 UTC (permalink / raw)
To: dtas-all
fiddle is distributed with Ruby since 1.9.2 (and the oldest we
support is 1.9.3). So instead of expecting potential users to
install Ruby development headers and a C compiler, just use
fiddle to save potential users bandwidth and storage space.
We have to define some hard-coded constants, but all of these
constants are architecture-independent and we can trust the
Linux kernel to never break userspace.
Eric Wong (5):
pipe: avoid loading sleepy_penguin
use fiddle-based eventfd implementation
buffer: replace sleepy_penguin with fiddle
watchable: use fiddle for inotify support
doc: remove "sleepy_penguin" references
Documentation/dtas-sinkedit.pod | 2 +-
Documentation/dtas-sourceedit.pod | 2 +-
INSTALL | 19 -----
dtas-linux.gemspec | 19 -----
lib/dtas.rb | 8 ++
lib/dtas/buffer.rb | 7 +-
.../buffer/{splice.rb => fiddle_splice.rb} | 75 ++++++++++++----
lib/dtas/buffer/read_write.rb | 4 +-
lib/dtas/pipe.rb | 20 +++--
lib/dtas/sigevent.rb | 3 +-
lib/dtas/sigevent/efd.rb | 20 -----
lib/dtas/sigevent/fiddle_efd.rb | 38 +++++++++
lib/dtas/sigevent/pipe.rb | 2 +-
lib/dtas/watchable.rb | 85 +++++++++++++++++--
test/test_buffer.rb | 10 +--
test/test_sigevent.rb | 20 +++++
test/test_sink_pipe_size.rb | 27 +++---
17 files changed, 240 insertions(+), 121 deletions(-)
delete mode 100644 dtas-linux.gemspec
rename lib/dtas/buffer/{splice.rb => fiddle_splice.rb} (68%)
delete mode 100644 lib/dtas/sigevent/efd.rb
create mode 100644 lib/dtas/sigevent/fiddle_efd.rb
create mode 100644 test/test_sigevent.rb
^ permalink raw reply [relevance 6%]
* [PATCH 4/5] watchable: use fiddle for inotify support
2019-12-01 1:26 6% [PATCH 0/5] use fiddle for Linux-specific APIs Eric Wong
@ 2019-12-01 1:26 5% ` Eric Wong
0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2019-12-01 1:26 UTC (permalink / raw)
To: dtas-all
We have String#unpack at our disposal for working with "struct
inotify_event", so use it instead of depending on an extension
which requires a compiler and development headers to install.
---
lib/dtas/watchable.rb | 85 +++++++++++++++++++++++++++++++++++++++----
1 file changed, 77 insertions(+), 8 deletions(-)
diff --git a/lib/dtas/watchable.rb b/lib/dtas/watchable.rb
index d0f37af..4047a42 100644
--- a/lib/dtas/watchable.rb
+++ b/lib/dtas/watchable.rb
@@ -1,22 +1,80 @@
# Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
+require_relative '../dtas'
+require_relative 'nonblock'
+
begin
-require 'sleepy_penguin'
+require 'fiddle'
# used to restart DTAS::Source::SplitFX processing in dtas-player
# if the YAML file is edited
module DTAS::Watchable # :nodoc:
- class InotifyReadableIter < SleepyPenguin::Inotify # :nodoc:
- def self.new
- super(:CLOEXEC)
+ class InotifyReadableIter # :nodoc:
+
+ Inotify_init = Fiddle::Function.new(DTAS.libc['inotify_init1'],
+ [ Fiddle::TYPE_INT ],
+ Fiddle::TYPE_INT)
+
+ Inotify_add_watch = Fiddle::Function.new(DTAS.libc['inotify_add_watch'],
+ [ Fiddle::TYPE_INT, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT ],
+ Fiddle::TYPE_INT)
+
+ # IO.select compatibility
+ attr_reader :to_io #:nodoc:
+
+ def initialize # :nodoc:
+ fd = Inotify_init.call(02000000 | 04000) # CLOEXEC | NONBLOCK
+ raise "inotify_init failed: #{Fiddle.last_error}" if fd < 0
+ @to_io = DTAS::Nonblock.for_fd(fd)
+ @buf = ''.b
+ @q = []
end
- FLAGS = CLOSE_WRITE | MOVED_TO
+ # struct inotify_event {
+ # int wd; /* Watch descriptor */
+ # uint32_t mask; /* Mask describing event */
+ # uint32_t cookie; /* Unique cookie associating related
+ # events (for rename(2)) */
+ # uint32_t len; /* Size of name field */
+ # char name[]; /* Optional null-terminated name */
+ InotifyEvent = Struct.new(:wd, :mask, :cookie, :len, :name) # :nodoc:
+
+ def take # :nodoc:
+ event = @q.pop and return event
+ case rv = @to_io.read_nonblock(16384, @buf, exception: false)
+ when :wait_readable, nil
+ return
+ else
+ until rv.empty?
+ hdr = rv.slice!(0,16)
+ name = nil
+ wd, mask, cookie, len = res = hdr.unpack('iIII')
+ wd && mask && cookie && len or
+ raise "bogus inotify_event #{res.inspect} hdr=#{hdr.inspect}"
+ if len > 0
+ name = rv.slice!(0, len)
+ name.size == len or raise "short name #{name.inspect} != #{len}"
+ name.sub!(/\0+\z/, '') or
+ raise "missing: `\\0', inotify_event.name=#{name.inspect}"
+ name = DTAS.dedupe_str(name)
+ end
+ ie = InotifyEvent.new(wd, mask, cookie, len, name)
+ if event
+ @q << ie
+ else
+ event = ie
+ end
+ end # /until rv.empty?
+ return event
+ end while true
+ end
+
+ FLAGS = 8 | 128 # CLOSE_WRITE | MOVED_TO
def readable_iter
or_call = false
- while event = take(true) # drain the buffer
+ while event = take # drain the buffer
w = @watches[event.wd] or next
if (event.mask & FLAGS) != 0 && w[event.name]
or_call = true
@@ -30,6 +88,16 @@ def readable_iter
end
end
+ def add_watch(watchdir)
+ wd = Inotify_add_watch.call(@to_io.fileno, watchdir, FLAGS)
+ raise "inotify_add_watch failed: #{Fiddle.last_error}" if wd < 0
+ wd
+ end
+
+ def close
+ @to_io = @to_io.close if @to_io
+ end
+
# we must watch the directory, since
def watch_files(paths, blk)
@watches = {} # wd -> { basename -> true }
@@ -38,7 +106,7 @@ def watch_files(paths, blk)
Array(paths).each do |path|
watchdir, watchbase = File.split(File.expand_path(path))
begin
- wd = @dir2wd[watchdir] ||= add_watch(watchdir, FLAGS)
+ wd = @dir2wd[watchdir] ||= add_watch(watchdir)
m = @watches[wd] ||= {}
m[watchbase] = true
rescue SystemCallError => e
@@ -67,5 +135,6 @@ def watch_end(srv)
end
end
-rescue LoadError
+rescue LoadError, Fiddle::DLError => e
+ warn "#{e.message} (#{e.class})"
end
^ permalink raw reply related [relevance 5%]
* [PATCH v2 0/5] support fiddle for Linux-only syscalls
@ 2019-12-20 1:39 7% Eric Wong
2019-12-20 1:39 4% ` [PATCH v2 4/5] watchable: use fiddle for inotify support Eric Wong
0 siblings, 1 reply; 5+ results
From: Eric Wong @ 2019-12-20 1:39 UTC (permalink / raw)
To: dtas-all
fiddle can be used as an alternative to sleepy_penguin, since
sleepy_penguin requires development headers and a compiler to
install, and fiddle is bundled as part of Ruby. sleepy_penguin
can still be faster, though, so we continue supporting
sleepy_penguin.
Eric Wong (5):
pipe: avoid loading sleepy_penguin
provide fiddle-based eventfd implementation
buffer: replace sleepy_penguin with fiddle
watchable: use fiddle for inotify support
doc: remove most recommendations for sleepy_penguin
Documentation/dtas-sinkedit.pod | 2 +-
Documentation/dtas-sourceedit.pod | 2 +-
INSTALL | 16 +--
lib/dtas.rb | 8 ++
lib/dtas/buffer.rb | 15 ++-
lib/dtas/buffer/fiddle_splice.rb | 216 ++++++++++++++++++++++++++++++
lib/dtas/buffer/read_write.rb | 4 +-
lib/dtas/buffer/splice.rb | 2 +
lib/dtas/pipe.rb | 20 +--
lib/dtas/sigevent.rb | 7 +-
lib/dtas/sigevent/efd.rb | 2 +
lib/dtas/sigevent/fiddle_efd.rb | 38 ++++++
lib/dtas/sigevent/pipe.rb | 2 +-
lib/dtas/watchable.rb | 115 ++++++++--------
lib/dtas/watchable/fiddle_ino.rb | 78 +++++++++++
lib/dtas/watchable/inotify.rb | 13 ++
test/test_buffer.rb | 10 +-
test/test_sigevent.rb | 20 +++
test/test_sink_pipe_size.rb | 27 ++--
19 files changed, 486 insertions(+), 111 deletions(-)
create mode 100644 lib/dtas/buffer/fiddle_splice.rb
create mode 100644 lib/dtas/sigevent/fiddle_efd.rb
create mode 100644 lib/dtas/watchable/fiddle_ino.rb
create mode 100644 lib/dtas/watchable/inotify.rb
create mode 100644 test/test_sigevent.rb
^ permalink raw reply [relevance 7%]
* [PATCH v2 4/5] watchable: use fiddle for inotify support
2019-12-20 1:39 7% [PATCH v2 0/5] support fiddle for Linux-only syscalls Eric Wong
@ 2019-12-20 1:39 4% ` Eric Wong
0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2019-12-20 1:39 UTC (permalink / raw)
To: dtas-all
We have String#unpack at our disposal for working with "struct
inotify_event", so use it instead of depending on an extension
which requires a compiler and development headers to install.
---
lib/dtas/watchable.rb | 115 ++++++++++++++++---------------
lib/dtas/watchable/fiddle_ino.rb | 78 +++++++++++++++++++++
lib/dtas/watchable/inotify.rb | 13 ++++
3 files changed, 149 insertions(+), 57 deletions(-)
create mode 100644 lib/dtas/watchable/fiddle_ino.rb
create mode 100644 lib/dtas/watchable/inotify.rb
diff --git a/lib/dtas/watchable.rb b/lib/dtas/watchable.rb
index d0f37af..2502b7c 100644
--- a/lib/dtas/watchable.rb
+++ b/lib/dtas/watchable.rb
@@ -1,71 +1,72 @@
# Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
+require_relative '../dtas'
+require_relative 'nonblock'
begin
-require 'sleepy_penguin'
+ module DTAS::Watchable # :nodoc:
+ module InotifyCommon # :nodoc:
+ FLAGS = 8 | 128 # IN_CLOSE_WRITE | IN_MOVED_TO
-# used to restart DTAS::Source::SplitFX processing in dtas-player
-# if the YAML file is edited
-module DTAS::Watchable # :nodoc:
- class InotifyReadableIter < SleepyPenguin::Inotify # :nodoc:
- def self.new
- super(:CLOEXEC)
- end
-
- FLAGS = CLOSE_WRITE | MOVED_TO
-
- def readable_iter
- or_call = false
- while event = take(true) # drain the buffer
- w = @watches[event.wd] or next
- if (event.mask & FLAGS) != 0 && w[event.name]
- or_call = true
+ def readable_iter
+ or_call = false
+ while event = take(true) # drain the buffer
+ w = @watches[event.wd] or next
+ if (event.mask & FLAGS) != 0 && w[event.name]
+ or_call = true
+ end
+ end
+ if or_call
+ @on_readable.call
+ :delete
+ else
+ :wait_readable
end
end
- if or_call
- @on_readable.call
- :delete
- else
- :wait_readable
- end
- end
- # we must watch the directory, since
- def watch_files(paths, blk)
- @watches = {} # wd -> { basename -> true }
- @on_readable = blk
- @dir2wd = {}
- Array(paths).each do |path|
- watchdir, watchbase = File.split(File.expand_path(path))
- begin
- wd = @dir2wd[watchdir] ||= add_watch(watchdir, FLAGS)
- m = @watches[wd] ||= {}
- m[watchbase] = true
- rescue SystemCallError => e
- warn "#{watchdir.dump}: #{e.message} (#{e.class})"
+ # we must watch the directory, since
+ def watch_files(paths, blk)
+ @watches = {} # wd -> { basename -> true }
+ @on_readable = blk
+ @dir2wd = {}
+ Array(paths).each do |path|
+ watchdir, watchbase = File.split(File.expand_path(path))
+ begin
+ wd = @dir2wd[watchdir] ||= add_watch(watchdir, FLAGS)
+ m = @watches[wd] ||= {}
+ m[watchbase] = true
+ rescue SystemCallError => e
+ warn "#{watchdir.dump}: #{e.message} (#{e.class})"
+ end
end
end
- end
- end
+ end # module InotifyCommon
- def watch_begin(blk)
- @ino = InotifyReadableIter.new
- @ino.watch_files(@watch_extra << @infile, blk)
- @ino
- end
+ begin
+ require_relative 'watchable/inotify'
+ rescue LoadError
+ # TODO: support kevent
+ require_relative 'watchable/fiddle_ino'
+ end
- def watch_extra(paths)
- @ino.watch_extra(paths)
- end
+ def watch_begin(blk)
+ @ino = DTAS::Watchable::InotifyReadableIter.new
+ @ino.watch_files(@watch_extra << @infile, blk)
+ @ino
+ end
- # Closing the inotify descriptor (instead of using inotify_rm_watch)
- # is cleaner because it avoids EINVAL on race conditions in case
- # a directory is deleted: https://lkml.org/lkml/2007/7/9/3
- def watch_end(srv)
- srv.wait_ctl(@ino, :delete)
- @ino = @ino.close
- end
-end
+ def watch_extra(paths)
+ @ino.watch_extra(paths)
+ end
-rescue LoadError
-end
+ # Closing the inotify descriptor (instead of using inotify_rm_watch)
+ # is cleaner because it avoids EINVAL on race conditions in case
+ # a directory is deleted: https://lkml.org/lkml/2007/7/9/3
+ def watch_end(srv)
+ srv.wait_ctl(@ino, :delete)
+ @ino = @ino.close
+ end
+ end # module DTAS::Watchable
+rescue LoadError, StandardError => e
+ warn "#{e.message} (#{e.class})"
+end # begin
diff --git a/lib/dtas/watchable/fiddle_ino.rb b/lib/dtas/watchable/fiddle_ino.rb
new file mode 100644
index 0000000..3b9d668
--- /dev/null
+++ b/lib/dtas/watchable/fiddle_ino.rb
@@ -0,0 +1,78 @@
+# Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# frozen_string_literal: true
+require 'fiddle'
+
+# used to restart DTAS::Source::SplitFX processing in dtas-player
+# if the YAML file is edited
+class DTAS::Watchable::InotifyReadableIter # :nodoc:
+ include DTAS::Watchable::InotifyCommon
+
+ Inotify_init = Fiddle::Function.new(DTAS.libc['inotify_init1'],
+ [ Fiddle::TYPE_INT ],
+ Fiddle::TYPE_INT)
+
+ Inotify_add_watch = Fiddle::Function.new(DTAS.libc['inotify_add_watch'],
+ [ Fiddle::TYPE_INT, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT ],
+ Fiddle::TYPE_INT)
+
+ # IO.select compatibility
+ attr_reader :to_io #:nodoc:
+
+ def initialize # :nodoc:
+ fd = Inotify_init.call(02000000 | 04000) # CLOEXEC | NONBLOCK
+ raise "inotify_init failed: #{Fiddle.last_error}" if fd < 0
+ @to_io = DTAS::Nonblock.for_fd(fd)
+ @buf = ''.b
+ @q = []
+ end
+
+ # struct inotify_event {
+ # int wd; /* Watch descriptor */
+ # uint32_t mask; /* Mask describing event */
+ # uint32_t cookie; /* Unique cookie associating related
+ # events (for rename(2)) */
+ # uint32_t len; /* Size of name field */
+ # char name[]; /* Optional null-terminated name */
+ InotifyEvent = Struct.new(:wd, :mask, :cookie, :len, :name) # :nodoc:
+
+ def take(nonblock) # :nodoc:
+ event = @q.pop and return event
+ case rv = @to_io.read_nonblock(16384, @buf, exception: false)
+ when :wait_readable, nil
+ return
+ else
+ until rv.empty?
+ hdr = rv.slice!(0,16)
+ name = nil
+ wd, mask, cookie, len = res = hdr.unpack('iIII')
+ wd && mask && cookie && len or
+ raise "bogus inotify_event #{res.inspect} hdr=#{hdr.inspect}"
+ if len > 0
+ name = rv.slice!(0, len)
+ name.size == len or raise "short name #{name.inspect} != #{len}"
+ name.sub!(/\0+\z/, '') or
+ raise "missing: `\\0', inotify_event.name=#{name.inspect}"
+ name = DTAS.dedupe_str(name)
+ end
+ ie = InotifyEvent.new(wd, mask, cookie, len, name)
+ if event
+ @q << ie
+ else
+ event = ie
+ end
+ end # /until rv.empty?
+ return event
+ end while true
+ end
+
+ def add_watch(watchdir, flags)
+ wd = Inotify_add_watch.call(@to_io.fileno, watchdir, flags)
+ raise "inotify_add_watch failed: #{Fiddle.last_error}" if wd < 0
+ wd
+ end
+
+ def close
+ @to_io = @to_io.close if @to_io
+ end
+end
diff --git a/lib/dtas/watchable/inotify.rb b/lib/dtas/watchable/inotify.rb
new file mode 100644
index 0000000..eface0e
--- /dev/null
+++ b/lib/dtas/watchable/inotify.rb
@@ -0,0 +1,13 @@
+# Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# frozen_string_literal: true
+require 'sleepy_penguin'
+
+# used to restart DTAS::Source::SplitFX processing in dtas-player
+# if the YAML file is edited
+class DTAS::Watchable::InotifyReadableIter < SleepyPenguin::Inotify # :nodoc:
+ include DTAS::Watchable::InotifyCommon
+ def self.new
+ super(:CLOEXEC)
+ end
+end
^ permalink raw reply related [relevance 4%]
* [ANN] dtas 0.18.0 - duct tape audio suite for *nix
@ 2020-02-03 8:36 6% Eric Wong
0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2020-02-03 8:36 UTC (permalink / raw)
To: dtas-all
Free Software command-line tools for audio playback, mastering, and
whatever else related to audio. dtas follows the worse-is-better
philosophy and acts as duct tape to combine existing command-line tools
for flexibility and ease-of-development. dtas is currently implemented
in Ruby (and some embedded shell), but may use other languages in the
future.
Changes:
dtas 0.18.0
fiddle is now supported for Linux users without a C compiler or
Ruby headers installed to take advantage of inotify(7)
functionality and efficiency improvements with splice(2) and
eventfd(7).
For non-RubyGems users, there's some setup.rb fixes thanks
to James Rowe.
Eric Wong (8):
doc: fixup leftover ftp://lists.gnu.org links
pipe: avoid loading sleepy_penguin
provide fiddle-based eventfd implementation
buffer: replace sleepy_penguin with fiddle
watchable: use fiddle for inotify support
doc: remove most recommendations for sleepy_penguin
INSTALL: s/bogomips.org/yhbt.net/
doc: update copyrights for 2020
James Rowe (2):
setup: update to use RbConfig
setup: fix duplicate variable warning
* homepage: https://80x24.org/dtas.git/about
* https://80x24.org/dtas/INSTALL
* https://80x24.org/dtas/dtas-player.txt
* https://80x24.org/dtas/NEWS.atom
* git clone https://80x24.org/dtas.git
* dtas-all@nongnu.org (plain-text only, no HTML mail, please)
* mail archives: https://80x24.org/dtas-all/
nntp://news.public-inbox.org/inbox.comp.audio.dtas
https://80x24.org/dtas-all/new.atom
^ permalink raw reply [relevance 6%]
Results 1-5 of 5 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-12-01 1:26 6% [PATCH 0/5] use fiddle for Linux-specific APIs Eric Wong
2019-12-01 1:26 5% ` [PATCH 4/5] watchable: use fiddle for inotify support Eric Wong
2019-12-20 1:39 7% [PATCH v2 0/5] support fiddle for Linux-only syscalls Eric Wong
2019-12-20 1:39 4% ` [PATCH v2 4/5] watchable: use fiddle for inotify support Eric Wong
2020-02-03 8:36 6% [ANN] dtas 0.18.0 - duct tape audio suite for *nix Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/dtas.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).