everything related to duct tape audio suite (dtas)
 help / color / mirror / code / Atom feed
Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [PATCH 0/5] use fiddle for Linux-specific APIs
@ 2019-12-01  1:26  6% Eric Wong
  2019-12-01  1:26  5% ` [PATCH 4/5] watchable: use fiddle for inotify support Eric Wong
  0 siblings, 1 reply; 5+ results
From: Eric Wong @ 2019-12-01  1:26 UTC (permalink / raw)
  To: dtas-all

fiddle is distributed with Ruby since 1.9.2 (and the oldest we
support is 1.9.3).  So instead of expecting potential users to
install Ruby development headers and a C compiler, just use
fiddle to save potential users bandwidth and storage space.

We have to define some hard-coded constants, but all of these
constants are architecture-independent and we can trust the
Linux kernel to never break userspace.

Eric Wong (5):
  pipe: avoid loading sleepy_penguin
  use fiddle-based eventfd implementation
  buffer: replace sleepy_penguin with fiddle
  watchable: use fiddle for inotify support
  doc: remove "sleepy_penguin" references

 Documentation/dtas-sinkedit.pod               |  2 +-
 Documentation/dtas-sourceedit.pod             |  2 +-
 INSTALL                                       | 19 -----
 dtas-linux.gemspec                            | 19 -----
 lib/dtas.rb                                   |  8 ++
 lib/dtas/buffer.rb                            |  7 +-
 .../buffer/{splice.rb => fiddle_splice.rb}    | 75 ++++++++++++----
 lib/dtas/buffer/read_write.rb                 |  4 +-
 lib/dtas/pipe.rb                              | 20 +++--
 lib/dtas/sigevent.rb                          |  3 +-
 lib/dtas/sigevent/efd.rb                      | 20 -----
 lib/dtas/sigevent/fiddle_efd.rb               | 38 +++++++++
 lib/dtas/sigevent/pipe.rb                     |  2 +-
 lib/dtas/watchable.rb                         | 85 +++++++++++++++++--
 test/test_buffer.rb                           | 10 +--
 test/test_sigevent.rb                         | 20 +++++
 test/test_sink_pipe_size.rb                   | 27 +++---
 17 files changed, 240 insertions(+), 121 deletions(-)
 delete mode 100644 dtas-linux.gemspec
 rename lib/dtas/buffer/{splice.rb => fiddle_splice.rb} (68%)
 delete mode 100644 lib/dtas/sigevent/efd.rb
 create mode 100644 lib/dtas/sigevent/fiddle_efd.rb
 create mode 100644 test/test_sigevent.rb



^ permalink raw reply	[relevance 6%]

* [PATCH 4/5] watchable: use fiddle for inotify support
  2019-12-01  1:26  6% [PATCH 0/5] use fiddle for Linux-specific APIs Eric Wong
@ 2019-12-01  1:26  5% ` Eric Wong
  0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2019-12-01  1:26 UTC (permalink / raw)
  To: dtas-all

We have String#unpack at our disposal for working with "struct
inotify_event", so use it instead of depending on an extension
which requires a compiler and development headers to install.
---
 lib/dtas/watchable.rb | 85 +++++++++++++++++++++++++++++++++++++++----
 1 file changed, 77 insertions(+), 8 deletions(-)

diff --git a/lib/dtas/watchable.rb b/lib/dtas/watchable.rb
index d0f37af..4047a42 100644
--- a/lib/dtas/watchable.rb
+++ b/lib/dtas/watchable.rb
@@ -1,22 +1,80 @@
 # Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 # frozen_string_literal: true
+require_relative '../dtas'
+require_relative 'nonblock'
+
 begin
-require 'sleepy_penguin'
+require 'fiddle'
 
 # used to restart DTAS::Source::SplitFX processing in dtas-player
 # if the YAML file is edited
 module DTAS::Watchable # :nodoc:
-  class InotifyReadableIter < SleepyPenguin::Inotify # :nodoc:
-    def self.new
-      super(:CLOEXEC)
+  class InotifyReadableIter # :nodoc:
+
+    Inotify_init = Fiddle::Function.new(DTAS.libc['inotify_init1'],
+      [ Fiddle::TYPE_INT ],
+      Fiddle::TYPE_INT)
+
+    Inotify_add_watch = Fiddle::Function.new(DTAS.libc['inotify_add_watch'],
+      [ Fiddle::TYPE_INT, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT ],
+      Fiddle::TYPE_INT)
+
+    # IO.select compatibility
+    attr_reader :to_io  #:nodoc:
+
+    def initialize # :nodoc:
+      fd = Inotify_init.call(02000000 | 04000) # CLOEXEC | NONBLOCK
+      raise "inotify_init failed: #{Fiddle.last_error}" if fd < 0
+      @to_io = DTAS::Nonblock.for_fd(fd)
+      @buf = ''.b
+      @q = []
     end
 
-    FLAGS = CLOSE_WRITE | MOVED_TO
+    # struct inotify_event {
+    #     int      wd;       /* Watch descriptor */
+    #     uint32_t mask;     /* Mask describing event */
+    #     uint32_t cookie;   /* Unique cookie associating related
+    #                           events (for rename(2)) */
+    #     uint32_t len;      /* Size of name field */
+    #     char     name[];   /* Optional null-terminated name */
+    InotifyEvent = Struct.new(:wd, :mask, :cookie, :len, :name) # :nodoc:
+
+    def take # :nodoc:
+      event = @q.pop and return event
+      case rv = @to_io.read_nonblock(16384, @buf, exception: false)
+      when :wait_readable, nil
+        return
+      else
+        until rv.empty?
+          hdr = rv.slice!(0,16)
+          name = nil
+          wd, mask, cookie, len = res = hdr.unpack('iIII')
+          wd && mask && cookie && len or
+            raise "bogus inotify_event #{res.inspect} hdr=#{hdr.inspect}"
+          if len > 0
+            name = rv.slice!(0, len)
+            name.size == len or raise "short name #{name.inspect} != #{len}"
+            name.sub!(/\0+\z/, '') or
+              raise "missing: `\\0', inotify_event.name=#{name.inspect}"
+            name = DTAS.dedupe_str(name)
+          end
+          ie = InotifyEvent.new(wd, mask, cookie, len, name)
+          if event
+            @q << ie
+          else
+            event = ie
+          end
+        end # /until rv.empty?
+        return event
+      end while true
+    end
+
+    FLAGS = 8 | 128 # CLOSE_WRITE | MOVED_TO
 
     def readable_iter
       or_call = false
-      while event = take(true) # drain the buffer
+      while event = take # drain the buffer
         w = @watches[event.wd] or next
         if (event.mask & FLAGS) != 0 && w[event.name]
           or_call = true
@@ -30,6 +88,16 @@ def readable_iter
       end
     end
 
+    def add_watch(watchdir)
+      wd = Inotify_add_watch.call(@to_io.fileno, watchdir, FLAGS)
+      raise "inotify_add_watch failed: #{Fiddle.last_error}" if wd < 0
+      wd
+    end
+
+    def close
+      @to_io = @to_io.close if @to_io
+    end
+
     # we must watch the directory, since
     def watch_files(paths, blk)
       @watches = {} # wd -> { basename -> true }
@@ -38,7 +106,7 @@ def watch_files(paths, blk)
       Array(paths).each do |path|
         watchdir, watchbase = File.split(File.expand_path(path))
         begin
-          wd = @dir2wd[watchdir] ||= add_watch(watchdir, FLAGS)
+          wd = @dir2wd[watchdir] ||= add_watch(watchdir)
           m = @watches[wd] ||= {}
           m[watchbase] = true
         rescue SystemCallError => e
@@ -67,5 +135,6 @@ def watch_end(srv)
   end
 end
 
-rescue LoadError
+rescue LoadError, Fiddle::DLError => e
+  warn "#{e.message} (#{e.class})"
 end


^ permalink raw reply related	[relevance 5%]

* [PATCH v2 0/5] support fiddle for Linux-only syscalls
@ 2019-12-20  1:39  7% Eric Wong
  2019-12-20  1:39  4% ` [PATCH v2 4/5] watchable: use fiddle for inotify support Eric Wong
  0 siblings, 1 reply; 5+ results
From: Eric Wong @ 2019-12-20  1:39 UTC (permalink / raw)
  To: dtas-all

fiddle can be used as an alternative to sleepy_penguin, since
sleepy_penguin requires development headers and a compiler to
install, and fiddle is bundled as part of Ruby.  sleepy_penguin
can still be faster, though, so we continue supporting
sleepy_penguin.

Eric Wong (5):
  pipe: avoid loading sleepy_penguin
  provide fiddle-based eventfd implementation
  buffer: replace sleepy_penguin with fiddle
  watchable: use fiddle for inotify support
  doc: remove most recommendations for sleepy_penguin

 Documentation/dtas-sinkedit.pod   |   2 +-
 Documentation/dtas-sourceedit.pod |   2 +-
 INSTALL                           |  16 +--
 lib/dtas.rb                       |   8 ++
 lib/dtas/buffer.rb                |  15 ++-
 lib/dtas/buffer/fiddle_splice.rb  | 216 ++++++++++++++++++++++++++++++
 lib/dtas/buffer/read_write.rb     |   4 +-
 lib/dtas/buffer/splice.rb         |   2 +
 lib/dtas/pipe.rb                  |  20 +--
 lib/dtas/sigevent.rb              |   7 +-
 lib/dtas/sigevent/efd.rb          |   2 +
 lib/dtas/sigevent/fiddle_efd.rb   |  38 ++++++
 lib/dtas/sigevent/pipe.rb         |   2 +-
 lib/dtas/watchable.rb             | 115 ++++++++--------
 lib/dtas/watchable/fiddle_ino.rb  |  78 +++++++++++
 lib/dtas/watchable/inotify.rb     |  13 ++
 test/test_buffer.rb               |  10 +-
 test/test_sigevent.rb             |  20 +++
 test/test_sink_pipe_size.rb       |  27 ++--
 19 files changed, 486 insertions(+), 111 deletions(-)
 create mode 100644 lib/dtas/buffer/fiddle_splice.rb
 create mode 100644 lib/dtas/sigevent/fiddle_efd.rb
 create mode 100644 lib/dtas/watchable/fiddle_ino.rb
 create mode 100644 lib/dtas/watchable/inotify.rb
 create mode 100644 test/test_sigevent.rb



^ permalink raw reply	[relevance 7%]

* [PATCH v2 4/5] watchable: use fiddle for inotify support
  2019-12-20  1:39  7% [PATCH v2 0/5] support fiddle for Linux-only syscalls Eric Wong
@ 2019-12-20  1:39  4% ` Eric Wong
  0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2019-12-20  1:39 UTC (permalink / raw)
  To: dtas-all

We have String#unpack at our disposal for working with "struct
inotify_event", so use it instead of depending on an extension
which requires a compiler and development headers to install.
---
 lib/dtas/watchable.rb            | 115 ++++++++++++++++---------------
 lib/dtas/watchable/fiddle_ino.rb |  78 +++++++++++++++++++++
 lib/dtas/watchable/inotify.rb    |  13 ++++
 3 files changed, 149 insertions(+), 57 deletions(-)
 create mode 100644 lib/dtas/watchable/fiddle_ino.rb
 create mode 100644 lib/dtas/watchable/inotify.rb

diff --git a/lib/dtas/watchable.rb b/lib/dtas/watchable.rb
index d0f37af..2502b7c 100644
--- a/lib/dtas/watchable.rb
+++ b/lib/dtas/watchable.rb
@@ -1,71 +1,72 @@
 # Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
 # License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
 # frozen_string_literal: true
+require_relative '../dtas'
+require_relative 'nonblock'
 begin
-require 'sleepy_penguin'
+  module DTAS::Watchable # :nodoc:
+    module InotifyCommon # :nodoc:
+      FLAGS = 8 | 128 # IN_CLOSE_WRITE | IN_MOVED_TO
 
-# used to restart DTAS::Source::SplitFX processing in dtas-player
-# if the YAML file is edited
-module DTAS::Watchable # :nodoc:
-  class InotifyReadableIter < SleepyPenguin::Inotify # :nodoc:
-    def self.new
-      super(:CLOEXEC)
-    end
-
-    FLAGS = CLOSE_WRITE | MOVED_TO
-
-    def readable_iter
-      or_call = false
-      while event = take(true) # drain the buffer
-        w = @watches[event.wd] or next
-        if (event.mask & FLAGS) != 0 && w[event.name]
-          or_call = true
+      def readable_iter
+        or_call = false
+        while event = take(true) # drain the buffer
+          w = @watches[event.wd] or next
+          if (event.mask & FLAGS) != 0 && w[event.name]
+            or_call = true
+          end
+        end
+        if or_call
+          @on_readable.call
+          :delete
+        else
+          :wait_readable
         end
       end
-      if or_call
-        @on_readable.call
-        :delete
-      else
-        :wait_readable
-      end
-    end
 
-    # we must watch the directory, since
-    def watch_files(paths, blk)
-      @watches = {} # wd -> { basename -> true }
-      @on_readable = blk
-      @dir2wd = {}
-      Array(paths).each do |path|
-        watchdir, watchbase = File.split(File.expand_path(path))
-        begin
-          wd = @dir2wd[watchdir] ||= add_watch(watchdir, FLAGS)
-          m = @watches[wd] ||= {}
-          m[watchbase] = true
-        rescue SystemCallError => e
-          warn "#{watchdir.dump}: #{e.message} (#{e.class})"
+      # we must watch the directory, since
+      def watch_files(paths, blk)
+        @watches = {} # wd -> { basename -> true }
+        @on_readable = blk
+        @dir2wd = {}
+        Array(paths).each do |path|
+          watchdir, watchbase = File.split(File.expand_path(path))
+          begin
+            wd = @dir2wd[watchdir] ||= add_watch(watchdir, FLAGS)
+            m = @watches[wd] ||= {}
+            m[watchbase] = true
+          rescue SystemCallError => e
+            warn "#{watchdir.dump}: #{e.message} (#{e.class})"
+          end
         end
       end
-    end
-  end
+    end # module InotifyCommon
 
-  def watch_begin(blk)
-    @ino = InotifyReadableIter.new
-    @ino.watch_files(@watch_extra << @infile, blk)
-    @ino
-  end
+    begin
+      require_relative 'watchable/inotify'
+    rescue LoadError
+      # TODO: support kevent
+      require_relative 'watchable/fiddle_ino'
+    end
 
-  def watch_extra(paths)
-    @ino.watch_extra(paths)
-  end
+    def watch_begin(blk)
+      @ino = DTAS::Watchable::InotifyReadableIter.new
+      @ino.watch_files(@watch_extra << @infile, blk)
+      @ino
+    end
 
-  # Closing the inotify descriptor (instead of using inotify_rm_watch)
-  # is cleaner because it avoids EINVAL on race conditions in case
-  # a directory is deleted: https://lkml.org/lkml/2007/7/9/3
-  def watch_end(srv)
-    srv.wait_ctl(@ino, :delete)
-    @ino = @ino.close
-  end
-end
+    def watch_extra(paths)
+      @ino.watch_extra(paths)
+    end
 
-rescue LoadError
-end
+    # Closing the inotify descriptor (instead of using inotify_rm_watch)
+    # is cleaner because it avoids EINVAL on race conditions in case
+    # a directory is deleted: https://lkml.org/lkml/2007/7/9/3
+    def watch_end(srv)
+      srv.wait_ctl(@ino, :delete)
+      @ino = @ino.close
+    end
+  end # module DTAS::Watchable
+rescue LoadError, StandardError => e
+  warn "#{e.message} (#{e.class})"
+end # begin
diff --git a/lib/dtas/watchable/fiddle_ino.rb b/lib/dtas/watchable/fiddle_ino.rb
new file mode 100644
index 0000000..3b9d668
--- /dev/null
+++ b/lib/dtas/watchable/fiddle_ino.rb
@@ -0,0 +1,78 @@
+# Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# frozen_string_literal: true
+require 'fiddle'
+
+# used to restart DTAS::Source::SplitFX processing in dtas-player
+# if the YAML file is edited
+class DTAS::Watchable::InotifyReadableIter # :nodoc:
+  include DTAS::Watchable::InotifyCommon
+
+  Inotify_init = Fiddle::Function.new(DTAS.libc['inotify_init1'],
+    [ Fiddle::TYPE_INT ],
+    Fiddle::TYPE_INT)
+
+  Inotify_add_watch = Fiddle::Function.new(DTAS.libc['inotify_add_watch'],
+    [ Fiddle::TYPE_INT, Fiddle::TYPE_VOIDP, Fiddle::TYPE_INT ],
+    Fiddle::TYPE_INT)
+
+  # IO.select compatibility
+  attr_reader :to_io  #:nodoc:
+
+  def initialize # :nodoc:
+    fd = Inotify_init.call(02000000 | 04000) # CLOEXEC | NONBLOCK
+    raise "inotify_init failed: #{Fiddle.last_error}" if fd < 0
+    @to_io = DTAS::Nonblock.for_fd(fd)
+    @buf = ''.b
+    @q = []
+  end
+
+  # struct inotify_event {
+  #     int      wd;       /* Watch descriptor */
+  #     uint32_t mask;     /* Mask describing event */
+  #     uint32_t cookie;   /* Unique cookie associating related
+  #                           events (for rename(2)) */
+  #     uint32_t len;      /* Size of name field */
+  #     char     name[];   /* Optional null-terminated name */
+  InotifyEvent = Struct.new(:wd, :mask, :cookie, :len, :name) # :nodoc:
+
+  def take(nonblock) # :nodoc:
+    event = @q.pop and return event
+    case rv = @to_io.read_nonblock(16384, @buf, exception: false)
+    when :wait_readable, nil
+      return
+    else
+      until rv.empty?
+        hdr = rv.slice!(0,16)
+        name = nil
+        wd, mask, cookie, len = res = hdr.unpack('iIII')
+        wd && mask && cookie && len or
+          raise "bogus inotify_event #{res.inspect} hdr=#{hdr.inspect}"
+        if len > 0
+          name = rv.slice!(0, len)
+          name.size == len or raise "short name #{name.inspect} != #{len}"
+          name.sub!(/\0+\z/, '') or
+            raise "missing: `\\0', inotify_event.name=#{name.inspect}"
+          name = DTAS.dedupe_str(name)
+        end
+        ie = InotifyEvent.new(wd, mask, cookie, len, name)
+        if event
+          @q << ie
+        else
+          event = ie
+        end
+      end # /until rv.empty?
+      return event
+    end while true
+  end
+
+  def add_watch(watchdir, flags)
+    wd = Inotify_add_watch.call(@to_io.fileno, watchdir, flags)
+    raise "inotify_add_watch failed: #{Fiddle.last_error}" if wd < 0
+    wd
+  end
+
+  def close
+    @to_io = @to_io.close if @to_io
+  end
+end
diff --git a/lib/dtas/watchable/inotify.rb b/lib/dtas/watchable/inotify.rb
new file mode 100644
index 0000000..eface0e
--- /dev/null
+++ b/lib/dtas/watchable/inotify.rb
@@ -0,0 +1,13 @@
+# Copyright (C) 2013-2019 all contributors <dtas-all@nongnu.org>
+# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
+# frozen_string_literal: true
+require 'sleepy_penguin'
+
+# used to restart DTAS::Source::SplitFX processing in dtas-player
+# if the YAML file is edited
+class DTAS::Watchable::InotifyReadableIter < SleepyPenguin::Inotify # :nodoc:
+  include DTAS::Watchable::InotifyCommon
+  def self.new
+    super(:CLOEXEC)
+  end
+end


^ permalink raw reply related	[relevance 4%]

* [ANN] dtas 0.18.0 - duct tape audio suite for *nix
@ 2020-02-03  8:36  6% Eric Wong
  0 siblings, 0 replies; 5+ results
From: Eric Wong @ 2020-02-03  8:36 UTC (permalink / raw)
  To: dtas-all

Free Software command-line tools for audio playback, mastering, and
whatever else related to audio.  dtas follows the worse-is-better
philosophy and acts as duct tape to combine existing command-line tools
for flexibility and ease-of-development.  dtas is currently implemented
in Ruby (and some embedded shell), but may use other languages in the
future.

Changes:

    dtas 0.18.0
    
    fiddle is now supported for Linux users without a C compiler or
    Ruby headers installed to take advantage of inotify(7)
    functionality and efficiency improvements with splice(2) and
    eventfd(7).
    
    For non-RubyGems users, there's some setup.rb fixes thanks
    to James Rowe.
    
    Eric Wong (8):
          doc: fixup leftover ftp://lists.gnu.org links
          pipe: avoid loading sleepy_penguin
          provide fiddle-based eventfd implementation
          buffer: replace sleepy_penguin with fiddle
          watchable: use fiddle for inotify support
          doc: remove most recommendations for sleepy_penguin
          INSTALL: s/bogomips.org/yhbt.net/
          doc: update copyrights for 2020
    
    James Rowe (2):
          setup: update to use RbConfig
          setup: fix duplicate variable warning

* homepage: https://80x24.org/dtas.git/about
* https://80x24.org/dtas/INSTALL
* https://80x24.org/dtas/dtas-player.txt
* https://80x24.org/dtas/NEWS.atom
* git clone https://80x24.org/dtas.git
* dtas-all@nongnu.org (plain-text only, no HTML mail, please)
* mail archives: https://80x24.org/dtas-all/
  nntp://news.public-inbox.org/inbox.comp.audio.dtas
  https://80x24.org/dtas-all/new.atom


^ permalink raw reply	[relevance 6%]

Results 1-5 of 5 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2019-12-01  1:26  6% [PATCH 0/5] use fiddle for Linux-specific APIs Eric Wong
2019-12-01  1:26  5% ` [PATCH 4/5] watchable: use fiddle for inotify support Eric Wong
2019-12-20  1:39  7% [PATCH v2 0/5] support fiddle for Linux-only syscalls Eric Wong
2019-12-20  1:39  4% ` [PATCH v2 4/5] watchable: use fiddle for inotify support Eric Wong
2020-02-03  8:36  6% [ANN] dtas 0.18.0 - duct tape audio suite for *nix Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/dtas.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).