everything related to duct tape audio suite (dtas)
 help / color / mirror / code / Atom feed
 Warning: Initial query:
 %22mlib: use flock to get around SQLite busy errors%22
 returned no results, used:
 "mlib: use flock to get around SQLite busy errors"
 instead

Search results ordered by [date|relevance]  view[summary|nested|Atom feed]
thread overview below | download mbox.gz: |
* [ANN] dtas 0.16.0 - duct tape audio suite for *nix
@ 2019-01-02 21:35  5% Eric Wong
  0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2019-01-02 21:35 UTC (permalink / raw)
  To: ruby-talk, dtas-all

Free Software command-line tools for audio playback, mastering, and
whatever else related to audio.  dtas follows the worse-is-better
philosophy and acts as duct tape to combine existing command-line tools
for flexibility and ease-of-development.  dtas is currently implemented
in Ruby (and some embedded shell), but may use other languages in the
future.

Changes:

    A bunch of minor fixes and cleanups accumulating for the past
    two years since the last release.  It's tough to remember to
    make releases when I'm always running the latest version from
    git :x

    Most notably, "io_splice" is no longer used for dtas-linux
    users since "sleepy_penguin" includes all the functionality
    we use.  This is to reduce memory overhead from extra DSOs(*)

    There's also some deprecation warning fixes for the
    still-undocumented "dtas-mlib" command.

    12 changes since v0.15.0 (2017-04-07):

          pipeline: new module for running process pipelines
          console: ensure time calculations are done in UTC
          Rakefile: update path for uploads
          player: support guessing encodings for comments
          get rid of Windows-31J regexps
          mlib: compatibility with Sequel 5.x
          mlib: remove redundant tag massaging and encoding
          mlib: use flock to get around SQLite busy errors
          mlib: ignore files with nil times
          dtas/watchable: check SystemCallError
          mlib: fix unused variable warning
          use sleepy_penguin 3.5+ for splice and tee support

    (*) https://udrepper.livejournal.com/8790.html

* homepage: https://80x24.org/dtas/README
* https://80x24.org/dtas/INSTALL
* https://80x24.org/dtas/dtas-player.txt
* https://80x24.org/dtas/NEWS.atom
* git clone https://80x24.org/dtas.git
* dtas-all@nongnu.org (plain-text only, no HTML mail, please)
* mailing list archives: https://80x24.org/dtas-all/
  nntp://news.public-inbox.org/inbox.comp.audio.dtas
  https://80x24.org/dtas-all/new.atom


^ permalink raw reply	[relevance 5%]

* [PATCH 3/4] mlib: use flock to get around SQLite busy errors
  2018-01-30  9:17  6% [PATCH 0/4] mlib: misc updates Eric Wong
@ 2018-01-30  9:17  7% ` Eric Wong
  0 siblings, 0 replies; 3+ results
From: Eric Wong @ 2018-01-30  9:17 UTC (permalink / raw)
  To: dtas-all

The SQLite busy waiting scheme isn't great for usability
and the busy timeout is done by sleep+backoff.  I prefer
to say we only support filesystems with flock() for our
little DB.
---
 lib/dtas/mlib.rb | 33 ++++++++++++++++++++++++++-------
 1 file changed, 26 insertions(+), 7 deletions(-)

diff --git a/lib/dtas/mlib.rb b/lib/dtas/mlib.rb
index d1707fb..3f5763f 100644
--- a/lib/dtas/mlib.rb
+++ b/lib/dtas/mlib.rb
@@ -10,6 +10,7 @@
 require_relative 'source/ff'
 require_relative 'source/splitfx'
 require 'socket'
+require 'tempfile'
 
 # For the DTAS Music Library, based on what MPD uses.
 class DTAS::Mlib # :nodoc:
@@ -62,15 +63,24 @@ def initialize(db)
     ]
   end
 
+  def synchronize
+    @lock.flock(File::LOCK_EX)
+    @db.transaction { yield }
+  ensure
+    @lock.flock(File::LOCK_UN)
+  end
+
   def init_suffixes
     `sox --help 2>/dev/null` =~ /\nAUDIO FILE FORMATS:\s*([^\n]+)/
     re = $1.split(/\s+/).map! { |x| Regexp.quote(x) }.join('|')
     @suffixes = Regexp.new("\\.(?:#{re}|yml)\\z", Regexp::IGNORECASE)
   end
 
-  def worker(todo)
+  def worker(todo, lock)
+    old_lock = @lock
+    @lock = lock
     @work.close
-    @db.tables # reconnect before chdir
+    synchronize { @db.tables } # reconnect before chdir
     @pwd = Dir.pwd.b
     begin
       buf = todo.recv(16384) # 4x bigger than PATH_MAX ought to be enough
@@ -84,7 +94,7 @@ def worker(todo)
   end
 
   def ignore(job)
-    @db.transaction do
+    synchronize do
       node_ensure(job.parent_id, job.path, DM_IGN, job.ctime)
     end
   end
@@ -112,7 +122,7 @@ def worker_work(job)
         tmp[tag_id] = value if value.valid_encoding?
       end
     end
-    @db.transaction do
+    synchronize do
       node_id = node_ensure(job.parent_id, path, tlen, job.ctime)[:id]
       vals = @db[:vals]
       comments = @db[:comments]
@@ -143,11 +153,20 @@ def update(path, opts = nil)
     @work and raise 'update already running'
     todo, @work = UNIXSocket.pair(:SOCK_SEQPACKET)
     @db.disconnect
-    jobs.times { |i| fork { worker(todo) } }
+
+    # like a Mutex between processes
+    @lock = Tempfile.new('dtas.mlib.lock')
+    jobs.times do |i|
+      lock = File.open(@lock.path, 'w')
+      fork { worker(todo, lock) }
+      lock.close
+    end
+    @lock.unlink
     todo.close
     scan_dir(path, st)
     @work.close
     Process.waitall
+    @lock.close
   ensure
     @work = nil
   end
@@ -235,7 +254,7 @@ def root_node
   end
 
   def dir_vivify(parts, ctime)
-    @db.transaction do
+    synchronize do
       dir = root_node
       last = parts.pop
       parts.each do |name|
@@ -289,7 +308,7 @@ def scan_dir(path, st, parent_id = nil)
       dir = dir_vivify(@pwd.split(%r{/+}n), st.ctime.to_i)
       dir_id = dir[:id]
 
-      @db.transaction do
+      synchronize do
         @db[:nodes].where(parent_id: dir_id).each do |node|
           File.exist?(node[:name]) or remove_entry(node)
         end
-- 
EW



^ permalink raw reply related	[relevance 7%]

* [PATCH 0/4] mlib: misc updates
@ 2018-01-30  9:17  6% Eric Wong
  2018-01-30  9:17  7% ` [PATCH 3/4] mlib: use flock to get around SQLite busy errors Eric Wong
  0 siblings, 1 reply; 3+ results
From: Eric Wong @ 2018-01-30  9:17 UTC (permalink / raw)
  To: dtas-all

Admittedly, I haven't used mlib in a while, so there was some
bitrot.  However, Sequel 5.x is out and it can benefit from
the encoding tweaks I did the other day.

Eric Wong (4):
      mlib: compatibility with Sequel 5.x
      mlib: remove redundant tag massaging and encoding
      mlib: use flock to get around SQLite busy errors
      mlib: ignore files with nil times

 lib/dtas/mlib.rb  | 66 +++++++++++++++++++++++++++++++------------------------
 test/test_mlib.rb |  4 ++--
 2 files changed, 39 insertions(+), 31 deletions(-)



^ permalink raw reply	[relevance 6%]

Results 1-3 of 3 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2018-01-30  9:17  6% [PATCH 0/4] mlib: misc updates Eric Wong
2018-01-30  9:17  7% ` [PATCH 3/4] mlib: use flock to get around SQLite busy errors Eric Wong
2019-01-02 21:35  5% [ANN] dtas 0.16.0 - duct tape audio suite for *nix Eric Wong

Code repositories for project(s) associated with this public inbox

	https://80x24.org/dtas.git/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).