* [ANN] dtas 0.12.0 - duct tape audio suite for *nix
@ 2015-12-14 4:03 7% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2015-12-14 4:03 UTC (permalink / raw)
To: ruby-talk; +Cc: dtas-all
Free Software command-line tools for audio playback, mastering, and
whatever else related to audio. dtas follows the worse-is-better
philosophy and acts as duct tape to combine existing command-line tools
for flexibility and ease-of-development. dtas is currently implemented
in Ruby (and some embedded shell), but may use other languages in the
future.
Changes: dtas 0.12.0 - many player updates
Notable additions for dtas-player music listeners.
dtas-ctl queue cat - display the internal queue
dtas-tl shuffle [true|false] - toggle tracklist shuffle
dtas-tl edit - open tracklist in an editor
see dtas-tl(1) manpage for more details on this:
http://dtas.80x24.org/dtas-tl.txt
The player tracklist now uses sequential track IDs instead of Ruby
object IDs, so it should be easier to use existing commands such as
"dtas-tl goto" and "dtas-tl remove"
For people using dtas-player for applying real-time effects to
splitfx YAML files, dtas-player can now watch for changes in scripts
specified in the "command" field of the splitfx YAML file.
Previously, dtas-player could only read the splitfx YAML file
itself. This requires GNU/Linux and the sleepy_penguin RubyGem
installed.
dtas-splitfx also learned some switches to allow easier
interoperability with other processing tools:
-p/--sox-pipe - identical to the sox(1) option
-t/--trim - only expose part of the track, useful when
combined with the above -p switch
See http://dtas.80x24.org/dtas-splitfx.txt for more details.
There's also some work-in-progress stuff that's not well documented
at the moment. As with anything pre-1.0, expect the possibility
of incompatible changes.
Since I'm not good at designing protocols, I've also started working
on mpd compatibility layer to allow using the normal music playback
stuff with more clients (some of the clients I actually like, unlike
the seemingly GUI-only landscape of MPRIS :P).
Of course, dtas-player itself has most the capabilities of a
Unix shell; and those capabilities will certainly not be available
to mpd or MPRIS clients.
64 changes since dtas 0.11.0
INSTALL: update documentation for 0.11.0 release
splitfx: fix lossy output with player command is in use
splitfx: fix non-generic, user-specified targets
use monotonic clock on Ruby 2.1+
dtas-splitfx: no arguments for quiet and --no-dither
dtas-splitfx: support --trim argument
process: implement array expansion to preserve spaces
splitfx: show correct command for output
splitfx: remove support for encoding opus
dtas-console: bind "o" to display time in absolute seconds
splitfx: avoid double-truncation with user command
source/splitfx: allow watching extra external scripts
splitfx: drop unnecessary variable
splitfx: documentation for subclasses
splitfx: simplify output display
dtas-splitfx: comment describing -j (nothing) as infinite
dtas-*edit: fix inotify watch invocations
splitfx: allow -p/--sox-pipe option
splitfx: ensure rate is an integer
splitfx: set OUTFMT correctly for subenv if command is set
favor recv and recv_nonblock over recvmsg variants
parse_time: pass through numeric types
parse_freq: trivial new module for parsing frequencies
use a common /dev/null
player: add "queue cat" command
dtas-readahead: new script for -player users on Linux
dtas-readahead: avoid polling on pause
dtas-cueedit: escape path to temporary file
dtas-readahead: make executable
gemspec: use SPDX-compatible license
README: add link to NNTP and Atom feeds
player: cleanup command dispatch
dtas-archive: allow specifying SoX compression factor
gemspec: duplicate frozen string for older Rubygems
allow building the gem without pandoc
introduce dtas-mlib for music library functions
dtas-mlib: add dump support for debugging
tracklist: use lower number unique track IDs
tracklist: do not mutate @list when serializing
tracklist: avoid needlessly building a hash for track IDs
tracklist: shuffle support
tracklist: support limiting maximum tracklist size
player: refactor and document tracklist interface
player: tl (repeat|shuffle|max) and trim swap values
tracklist: fixup idempotent "tl shuffle false"
switch to exception-free non-blocking I/O
add .gitattributes for Ruby method detection
mlib: remove non-existent entries
mlib: add stats support
mlib: no kwargs for 1.9.3 compatibility
mlib: add find/search functionality based on mpd
mlib: split out the output format from the library
mlib: remove kwargs harder
player: reduce I/O priority of connected clients
mlib: SYSTEM_DEFAULT handlers for SIGINT and SIGPIPE
player: support "tl clear" internally
test_unixserver: remove test for element limit
player: dump state file asynchronously when requested
parse_time: enable frozen_string_literal
unix_accepted: enable checking for readability after emit
tracklist: swap functionality
player: show "tracklist" hash with summary info with "current"
dtas-tl: learn an "edit" sub command
doc: document additions to tracklist handling
* homepage: http://dtas.80x24.org/README
* http://dtas.80x24.org/INSTALL
* http://dtas.80x24.org/dtas-player.txt
* http://dtas.80x24.org/NEWS
* git clone git://80x24.org/dtas
* dtas-all@nongnu.org (plain-text only, no HTML mail, please)
* mailing list archives http://80x24.org/dtas-all/
--
EW
^ permalink raw reply [relevance 7%]
* [PATCH] introduce dtas-mlib for music library functions
@ 2015-11-27 11:50 4% Eric Wong
0 siblings, 0 replies; 2+ results
From: Eric Wong @ 2015-11-27 11:50 UTC (permalink / raw)
To: dtas-all
Eventually this will support searching and be the basis
of an mpd-compatible proxy in front of dtas-player
---
bin/dtas-mlib | 39 +++++
lib/dtas/mlib.rb | 290 +++++++++++++++++++++++++++++++
lib/dtas/mlib/migrations/0001_initial.rb | 42 +++++
test/test_mlib.rb | 31 ++++
4 files changed, 402 insertions(+)
create mode 100755 bin/dtas-mlib
create mode 100644 lib/dtas/mlib.rb
create mode 100644 lib/dtas/mlib/migrations/0001_initial.rb
create mode 100644 test/test_mlib.rb
diff --git a/bin/dtas-mlib b/bin/dtas-mlib
new file mode 100755
index 0000000..0e17d91
--- /dev/null
+++ b/bin/dtas-mlib
@@ -0,0 +1,39 @@
+#!/usr/bin/env ruby
+# Copyright (C) 2015 all contributors <dtas-all@nongnu.org>
+# License: GPLv3 or later (https://www.gnu.org/licenses/gpl-3.0.txt)
+usage = "#$0 [-d DATABASE-URI] ACTION [DIRECTORY]"
+Thread.abort_on_exception = $stderr.sync = $stdout.sync = true
+require 'dtas/mlib'
+require 'optparse'
+path = '~/.dtas/mlib.sqlite'
+db = File.expand_path(path)
+OptionParser.new('', 24, ' ') do |op|
+ op.banner = usage
+ op.on('-d', '--database <URI|PATH>', "database (default: #{path}") do |d|
+ db = d
+ end
+ op.on('-h', '--help') do
+ puts(op.to_s)
+ exit
+ end
+ op.parse!(ARGV)
+end
+
+unless db.include?('://')
+ dir = File.dirname(db)
+ unless File.directory?(dir)
+ require 'fileutils'
+ FileUtils.mkpath(dir)
+ end
+end
+
+case action = ARGV.shift
+when 'update'
+ directory = ARGV.shift or abort "DIRECTORY required\n#{usage}"
+else
+ abort usage
+end
+
+mlib = DTAS::Mlib.new(db)
+mlib.migrate
+mlib.__send__(action, directory)
diff --git a/lib/dtas/mlib.rb b/lib/dtas/mlib.rb
new file mode 100644
index 0000000..59a589c
--- /dev/null
+++ b/lib/dtas/mlib.rb
@@ -0,0 +1,290 @@
+# -*- encoding: utf-8 -*-
+# Copyright (C) 2015 all contributors <dtas-all@nongnu.org>
+# License: GPLv3 or later (https://www.gnu.org/licenses/gpl-3.0.txt)
+
+require_relative '../dtas'
+require_relative 'process'
+require 'socket'
+
+# For the DTAS Music Library, based on what MPD uses.
+class DTAS::Mlib
+ attr_accessor :follow_outside_symlinks
+ attr_accessor :follow_inside_symlinks
+ attr_accessor :tags
+ DM_DIR = -1
+ DM_IGN = -2
+ include DTAS::Process
+
+ Job = Struct.new(:wd, :ctime, :parent_id, :path)
+
+ # same capitalization as in mpd
+ TAGS = Hash[*(
+ %w(Artist ArtistSort
+ Album AlbumSort
+ AlbumArtist AlbumArtistSort
+ Title Track Name
+ Genre Date Composer Performer Comment Disc
+ MUSICBRAINZ_ARTISTID MUSICBRAINZ_ALBUMID
+ MUSICBRAINZ_ALBUMARTISTID
+ MUSICBRAINZ_TRACKID
+ MUSICBRAINZ_RELEASETRACKID).map! { |x| [ x.downcase, x ] }.flatten!)]
+
+ def initialize(db)
+ if String === db
+ db = "sqlite://#{db}" unless db.include?('://')
+ require 'sequel/no_core_ext'
+ db = Sequel.connect(db, single_threaded: true)
+ end
+ if db.class.to_s.downcase.include?('sqlite')
+ db.transaction_mode = :immediate
+ db.synchronous = :off
+ end
+ @db = db
+ @pwd = nil
+ @follow_outside_symlinks = true
+ @follow_inside_symlinks = true
+ @root_node = nil
+ @tags = TAGS.dup
+ @tag_map = nil
+ @suffixes = nil
+ @work = nil
+ end
+
+ def init_suffixes
+ `sox --help 2>/dev/null` =~ /\nAUDIO FILE FORMATS:\s*([^\n]+)/s
+ re = $1.split(/\s+/).map { |x| Regexp.quote(x) }.join('|')
+ @suffixes = Regexp.new("\\.(?:#{re})\\z", Regexp::IGNORECASE)
+ end
+
+ def worker(todo)
+ @work.close
+ @db.tables # reconnect before chdir
+ @pwd = Dir.pwd.b
+ begin
+ buf = todo.recv(16384) # 4x bigger than PATH_MAX ought to be enough
+ exit if buf.empty?
+ job = Marshal.load(buf)
+ buf.clear
+ worker_work(job)
+ rescue => e
+ warn "#{e.message} (#{e.class}) #{e.backtrace.join("\n")}\n"
+ end while true
+ end
+
+ def ignore(job)
+ @db.transaction do
+ node_ensure(job.parent_id, job.path, DM_IGN, job.ctime)
+ end
+ end
+
+ def worker_work(job)
+ tlen = nil
+ wd = job.wd
+ if wd != @pwd
+ Dir.chdir(wd)
+ @pwd = wd
+ end
+ tmp = {}
+ path = job.path
+ tlen = qx(%W(soxi -D #{path}), no_raise: true)
+ return ignore(job) unless String === tlen
+ tlen = tlen.to_f
+ return ignore(job) if tlen < 0
+ tlen = tlen.round
+ buf = qx(%W(soxi -a #{path}), no_raise: true)
+ return ignore(job) unless String === buf
+
+ # no, we don't support comments with newlines in them
+ buf = buf.split("\n".freeze)
+ while line = buf.shift
+ tag, value = line.split('='.freeze, 2)
+ tag && value or next
+ tag.downcase!
+ tag_id = @tag_map[tag] or next
+ value.strip!
+
+ # FIXME: this fallback needs testing
+ [ Encoding::UTF_8, Encoding::ISO_8859_1 ].each do |enc|
+ value.force_encoding(enc)
+ if value.valid_encoding?
+ value.encode!(Encoding::UTF_8) if enc != Encoding::UTF_8
+ tmp[tag_id] = value
+ break
+ end
+ end
+ end
+ @db.transaction do
+ node_id = node_ensure(job.parent_id, path, tlen, job.ctime)[:id]
+ vals = @db[:vals]
+ comments = @db[:comments]
+ q = { node_id: node_id }
+ comments.where(q).delete
+ tmp.each do |tid, val|
+ v = vals[val: val]
+ q[:val_id] = v ? v[:id] : vals.insert(val: val)
+ q[:tag_id] = tid
+ comments.insert(q)
+ end
+ end
+ end
+
+ def update(path, jobs: 8)
+ # n.b. "jobs" is for CPU concurrency. Audio media is typically stored
+ # on high-latency media or slow network file systems; so we use a high
+ # number of jobs by default to compensate for the seek-heavy workload
+ # this generates
+ init_suffixes
+ st = File.stat(path) # we always follow the first dir even if it's a symlink
+ st.directory? or
+ raise ArgumentError, "path: #{path.inspect} is not a directory"
+ @work and raise 'update already running'
+ todo, @work = UNIXSocket.pair(:SOCK_SEQPACKET)
+ @db.disconnect
+ jobs.times { |i| fork { worker(todo) } }
+ todo.close
+ scan_dir(path, st)
+ @work.close
+ Process.waitall
+ ensure
+ @work = nil
+ end
+
+ def migrate
+ require 'sequel'
+ Sequel.extension(:migration, :core_extensions) # ugh...
+ @db.transaction do
+ Sequel::Migrator.apply(@db, "#{File.dirname(__FILE__)}/mlib/migrations")
+ root_node # ensure this exists
+ load_tags
+ end
+ end
+
+ def load_tags
+ tag_map = {}
+ tags = @db[:tags]
+ @tags.each do |lc, mc|
+ unless q = tags[tag: mc]
+ q = { tag: mc }
+ q[:id] = tags.insert(q)
+ end
+ tag_map[lc] = q[:id]
+ end
+
+ # Xiph tags use "tracknumber" and "discnumber"
+ %w(track disc).each do |x|
+ tag_id = tag_map[x] and tag_map["#{x}number"] = tag_id
+ end
+ @tag_map = tag_map
+ end
+
+ def scan_any(path, parent_id)
+ st = File.lstat(path) rescue return
+ if st.directory?
+ scan_dir(path, st, parent_id)
+ elsif st.file?
+ scan_file(path, st, parent_id)
+ # elsif st.symlink? TODO
+ # scan_link(path, st, parent_id)
+ end
+ end
+
+ def scan_file(path, st, parent_id)
+ return if @suffixes !~ path || st.size == 0
+
+ # no-op if no change
+ if node = @db[:nodes][name: path, parent_id: parent_id]
+ return if st.ctime.to_i == node[:ctime] || node[:tlen] == DM_IGN
+ end
+
+ job = Job.new(@pwd, st.ctime.to_i, parent_id, path)
+ send_harder(@work, Marshal.dump(job))
+ end
+
+ def root_node
+ q = @root_node and return q
+ # root node always has parent_id: 1
+ q = {
+ parent_id: 1, # self
+ name: '',
+ }
+ node = @db[:nodes][q] and return (@root_node = node)
+ begin
+ q[:tlen] = DM_DIR
+ q[:id] = @db[:nodes].insert(q)
+ q
+ rescue Sequel::DatabaseError
+ # we may conflict on insert if we didn't use a transaction
+ raise if @db.in_transaction?
+ @root_node = @db[:paths][q] or raise
+ end
+ end
+
+ def dir_vivify(parts, ctime)
+ @db.transaction do
+ dir = root_node
+ last = parts.pop
+ parts.each do |name|
+ dir = node_ensure(dir[:id], name, DM_DIR)
+ end
+ node_ensure(dir[:id], last, DM_DIR, ctime)
+ end
+ end
+
+ def node_update_maybe(node, tlen, ctime)
+ q = {}
+ q[:ctime] = ctime if ctime && ctime != node[:ctime]
+ q[:tlen] = tlen if tlen != node[:tlen]
+ return if q.empty?
+ node_id = node.delete(:id)
+ @db[:nodes].where(id: node_id).update(node.merge(q))
+ node[:id] = node_id
+ end
+
+ def node_ensure(parent_id, name, tlen, ctime = nil)
+ q = { name: name, parent_id: parent_id }
+ if node = @db[:nodes][q]
+ node_update_maybe(node, tlen, ctime)
+ else
+ # brand new node
+ node = q.dup
+ node[:tlen] = tlen
+ node[:ctime] = ctime
+ node[:id] = @db[:nodes].insert(node)
+ end
+ node
+ end
+
+ def scan_dir(path, st, parent_id = nil)
+ prev_wd = @pwd
+ Dir.chdir(path)
+ cur = @pwd = Dir.pwd.b
+
+ # TODO: use parent_id if given
+ dir = dir_vivify(cur.split(%r{/+}n), st.ctime.to_i)
+ Dir.foreach('.', encoding: Encoding::BINARY) do |x|
+ case x
+ when '.', '..', %r{\n}n
+ # files with newlines in them are rare and last I checked (in 2008),
+ # mpd could not support them, either. So lets not bother for now.
+ next
+ else
+ scan_any(x, dir[:id])
+ end
+ end
+ ensure
+ Dir.chdir(prev_wd) if cur && prev_wd
+ @pwd = prev_wd
+ end
+
+ def send_harder(sock, msg)
+ sock.sendmsg(msg)
+ rescue Errno::EMSGSIZE
+ sock.setsockopt(:SOL_SOCKET, :SO_SNDBUF, msg.bytesize + 1024)
+ # if it still fails, oh well...
+ begin
+ sock.sendmsg(msg)
+ rescue => e
+ warn "#{msg.bytesize} too big, dropped #{e.class}"
+ end
+ end
+end
diff --git a/lib/dtas/mlib/migrations/0001_initial.rb b/lib/dtas/mlib/migrations/0001_initial.rb
new file mode 100644
index 0000000..f147cbe
--- /dev/null
+++ b/lib/dtas/mlib/migrations/0001_initial.rb
@@ -0,0 +1,42 @@
+# Copyright (C) 2015 all contributors <dtas-all@nongnu.org>
+# License: GPLv3 or later (https://www.gnu.org/licenses/gpl-3.0.txt)
+
+Sequel.migration do
+ up do
+ create_table(:nodes) do
+ primary_key :id
+ String :name, null: false # encoding: binary, POSIX
+ Integer :ctime
+ foreign_key :parent_id, :nodes, null: false # parent dir
+ # >= 0: tlen of track, -2: ignore, -1: directory
+ Integer :tlen, null: false
+ unique [ :parent_id, :name ]
+ end
+
+ create_table(:tags) do
+ primary_key :id
+ String :tag, null: false, unique: true # encoding: US-ASCII
+ end
+
+ create_table(:vals) do
+ primary_key :id
+ String :val, null: false, unique: true # encoding: UTF-8
+ end
+
+ create_table(:comments) do
+ foreign_key :node_id, :nodes, null: false
+ foreign_key :tag_id, :tags, null: false
+ foreign_key :val_id, :vals, null: false
+ primary_key [ :node_id, :tag_id, :val_id ]
+ index :node_id
+ index [ :tag_id, :val_id ]
+ end
+ end
+
+ down do
+ drop_table(:nodes)
+ drop_table(:tags)
+ drop_table(:vals)
+ drop_table(:comments)
+ end
+end
diff --git a/test/test_mlib.rb b/test/test_mlib.rb
new file mode 100644
index 0000000..072d1d0
--- /dev/null
+++ b/test/test_mlib.rb
@@ -0,0 +1,31 @@
+# Copyright (C) 2013-2015 all contributors <dtas-all@nongnu.org>
+# License: GPLv3 or later (https://www.gnu.org/licenses/gpl-3.0.txt)
+require_relative 'helper'
+begin
+ require 'dtas/mlib'
+ require 'sequel/no_core_ext'
+ require 'sqlite3'
+rescue LoadError => err
+ warn "skipping mlib test: #{err.message}"
+ exit 0
+end
+
+class TestMlib < Testcase
+ def setup
+ @db = Sequel.sqlite(':memory:')
+ end
+
+ def test_migrate
+ ml = DTAS::Mlib.new(@db)
+ begin
+ $-w = false
+ ml.migrate
+ tables = @db.tables
+ ensure
+ $-w = true
+ end
+ [ :nodes, :tags, :vals, :comments ].each do |t|
+ assert tables.include?(t), "missing #{t}"
+ end
+ end
+end
--
EW
^ permalink raw reply related [relevance 4%]
Results 1-2 of 2 | reverse | options above
-- pct% links below jump to the message on this page, permalinks otherwise --
2015-11-27 11:50 4% [PATCH] introduce dtas-mlib for music library functions Eric Wong
2015-12-14 4:03 7% [ANN] dtas 0.12.0 - duct tape audio suite for *nix Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/dtas.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).