* [PATCH 0/9] various cuts for memory savings
@ 2022-01-23 21:37 Eric Wong
2022-01-23 21:37 ` [PATCH 1/9] deduplicate and freeze pathnames + metadata Eric Wong
` (8 more replies)
0 siblings, 9 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
"dtas-tl prune" can simplify life a bit for users who frequently
replace/rename tracks on their FS (I do it while editing audio
with dtas-splitfx).
Moving to Ruby 2.3 lets us depend on a bunch of newer APIs and
compatibility code can be dropped, since YARV instructions for
old versions still remain. The overall difference isn't very
noticeable, but the diffstat showing deleted lines is nice.
Eric Wong (9):
deduplicate and freeze pathnames + metadata
player: remove omap conversion
dtas: drop unnecessary "require 'yaml'" statements
dtas-tl prune: cull missing files from tracklist
dtas-tl: drop encoding hacks, use binary stdout+stderr
use IO#wait_readable consistently
get rid of DTAS::Nonblock wrapper for Ruby <= 2.0
unix_accepted: drop Ruby < 2.3 support code
do not check IO#closed? before calling IO#close
Documentation/dtas-tl.pod | 4 ++-
bin/dtas-console | 1 -
bin/dtas-msinkctl | 1 -
bin/dtas-player | 3 +-
bin/dtas-readahead | 26 ++++-----------
bin/dtas-splitfx | 1 -
bin/dtas-tl | 55 +++++++++++++++++---------------
lib/dtas/buffer.rb | 4 +--
lib/dtas/buffer/read_write.rb | 3 +-
lib/dtas/edit_client.rb | 5 ++-
lib/dtas/mcache.rb | 4 +--
lib/dtas/nonblock.rb | 24 --------------
lib/dtas/pipe.rb | 5 ++-
lib/dtas/player.rb | 14 +++-----
lib/dtas/process.rb | 5 ++-
lib/dtas/sigevent/fiddle_efd.rb | 7 ++--
lib/dtas/sigevent/pipe.rb | 5 ++-
lib/dtas/sink.rb | 3 +-
lib/dtas/source/sox.rb | 2 +-
lib/dtas/source/splitfx.rb | 1 -
lib/dtas/track.rb | 2 +-
lib/dtas/unix_accepted.rb | 49 +++++++---------------------
lib/dtas/unix_client.rb | 4 +--
lib/dtas/unix_server.rb | 16 ++--------
lib/dtas/watchable.rb | 3 +-
lib/dtas/watchable/fiddle_ino.rb | 2 +-
test/player_integration.rb | 3 +-
test/test_buffer.rb | 2 +-
test/test_encoding.rb | 1 -
test/test_splitfx.rb | 1 -
test/test_tfx.rb | 1 -
test/test_unixserver.rb | 2 +-
32 files changed, 84 insertions(+), 175 deletions(-)
delete mode 100644 lib/dtas/nonblock.rb
^ permalink raw reply [flat|nested] 10+ messages in thread
* [PATCH 1/9] deduplicate and freeze pathnames + metadata
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 2/9] player: remove omap conversion Eric Wong
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
This ought to save some memory for dtas-player and
dtas-readahead users with multiple instances of the same
track(s) on their track list.
---
bin/dtas-readahead | 7 +++----
lib/dtas/edit_client.rb | 4 ++--
lib/dtas/mcache.rb | 4 ++--
lib/dtas/player.rb | 6 +++---
lib/dtas/sigevent/fiddle_efd.rb | 4 ++--
lib/dtas/source/sox.rb | 2 +-
lib/dtas/track.rb | 2 +-
7 files changed, 14 insertions(+), 15 deletions(-)
diff --git a/bin/dtas-readahead b/bin/dtas-readahead
index 3f46b95..996142c 100755
--- a/bin/dtas-readahead
+++ b/bin/dtas-readahead
@@ -141,7 +141,7 @@ def do_open(path)
buf << fp.read(fp.size - 4)
Dir.chdir(File.dirname(path)) do
yml = DTAS.yaml_load(buf)
- x = yml['infile'] and return File.open(File.expand_path(x).freeze)
+ x = yml['infile'] and return File.open(-File.expand_path(x))
end
end
end
@@ -158,9 +158,8 @@ def do_open(path)
cur = DTAS.yaml_load(c.req('current'))
while @todo_ra > 0 && fp.nil?
if current = cur['current']
- track = current['infile']
+ track = -current['infile']
break unless track.kind_of?(String)
- track.freeze
fp = work[track] ||= do_open(track)
cur_pid = current['pid']
if fp
@@ -197,7 +196,7 @@ def do_open(path)
repeat = c.req('tl repeat').split[-1]
while @todo_ra > 0 && idx && (cid = ids[idx])
fp = nil
- track = c.req("tl get #{cid}").sub!(/\A1 \d+=/, '').freeze
+ track = -(c.req("tl get #{cid}").sub!(/\A1 \d+=/, ''))
begin
fp = work[track] ||= do_open(track)
rescue SystemCallError
diff --git a/lib/dtas/edit_client.rb b/lib/dtas/edit_client.rb
index a885060..036d036 100644
--- a/lib/dtas/edit_client.rb
+++ b/lib/dtas/edit_client.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'tempfile'
@@ -14,7 +14,7 @@ def editor
v.empty? and next
return v
end
- 'vi'.freeze
+ 'vi'
end
def client_socket
diff --git a/lib/dtas/mcache.rb b/lib/dtas/mcache.rb
index 817bfb8..4f1e9e8 100644
--- a/lib/dtas/mcache.rb
+++ b/lib/dtas/mcache.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2016-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
# encoding: binary
@@ -22,7 +22,7 @@ def lookup(infile)
@tbl[bucket] = begin
cur = cur ? cur.clear : {}
if ret = yield(infile, cur)
- ret[:infile] = infile.frozen? ? infile : infile.dup.freeze
+ ret[:infile] = infile.frozen? ? infile : -(infile.dup)
ret[:btime] = DTAS.now
end
ret
diff --git a/lib/dtas/player.rb b/lib/dtas/player.rb
index ff7440f..243f0b0 100644
--- a/lib/dtas/player.rb
+++ b/lib/dtas/player.rb
@@ -208,13 +208,13 @@ def client_iter(io, msg)
command = msg.shift
case command
when "enq"
- enq_handler(io, msg[0])
+ enq_handler(io, -msg[0])
when "enq-cmd"
- enq_handler(io, { "command" => msg[0]})
+ enq_handler(io, { "command" => -msg[0]})
when "pause", "play", "play_pause"
play_pause_handler(io, command)
when "pwd"
- io.emit(Dir.pwd)
+ io.emit(-Dir.pwd)
else
m = "dpc_#{command.tr('-', '_')}"
__send__(m, io, msg) if respond_to?(m)
diff --git a/lib/dtas/sigevent/fiddle_efd.rb b/lib/dtas/sigevent/fiddle_efd.rb
index 40cec77..ec9ec65 100644
--- a/lib/dtas/sigevent/fiddle_efd.rb
+++ b/lib/dtas/sigevent/fiddle_efd.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
@@ -13,7 +13,7 @@ class DTAS::Sigevent # :nodoc:
Fiddle::TYPE_INT) # fd
attr_reader :to_io
- ONE = [ 1 ].pack('Q').freeze
+ ONE = -([ 1 ].pack('Q'))
def initialize
fd = EventFD.call(0, 02000000|00004000) # EFD_CLOEXEC|EFD_NONBLOCK
diff --git a/lib/dtas/source/sox.rb b/lib/dtas/source/sox.rb
index e38e23c..6ca29bc 100644
--- a/lib/dtas/source/sox.rb
+++ b/lib/dtas/source/sox.rb
@@ -63,7 +63,7 @@ def mcache_lookup(infile)
comments.each do |k,v|
v.chomp!
DTAS.try_enc(v, enc)
- comments[k] = v
+ comments[k] = -v
end
end
dst
diff --git a/lib/dtas/track.rb b/lib/dtas/track.rb
index 85b667a..3f4b813 100644
--- a/lib/dtas/track.rb
+++ b/lib/dtas/track.rb
@@ -9,6 +9,6 @@ class DTAS::Track # :nodoc:
def initialize(track_id, path)
@track_id = track_id
- @to_path = path
+ @to_path = -path
end
end
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 2/9] player: remove omap conversion
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
2022-01-23 21:37 ` [PATCH 1/9] deduplicate and freeze pathnames + metadata Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 3/9] dtas: drop unnecessary "require 'yaml'" statements Eric Wong
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
"omap" is specific to Ruby and makes interopability with other
languages more difficult.
While it's true environment variables are stored as an ordered
array of C strings (see environ(7)); order doesn't matter in
practice. Everyone in the real world treats the environment as
an unordered key-value store.
Followup-to: cac63517e7f751cc (doc: drop ordered map from examples, 2022-01-07)
---
lib/dtas/player.rb | 7 -------
1 file changed, 7 deletions(-)
diff --git a/lib/dtas/player.rb b/lib/dtas/player.rb
index 243f0b0..3db15c3 100644
--- a/lib/dtas/player.rb
+++ b/lib/dtas/player.rb
@@ -1,7 +1,6 @@
# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
-require 'yaml'
require 'shellwords'
require_relative '../dtas'
require_relative 'xs'
@@ -123,10 +122,6 @@ def to_hsh
rv
end
- def to_omap(hash)
- YAML::Omap === hash ? hash : YAML::Omap.new.merge!(hash)
- end
-
def self.load(hash)
rv = new
rv.instance_eval do
@@ -157,7 +152,6 @@ def self.load(hash)
@source_map.each do |name, src|
src_hsh = v[name] or next
src.load!(src_hsh)
- src.env = to_omap(src.env)
end
source_map_reload
end
@@ -170,7 +164,6 @@ def self.load(hash)
sinks.each do |sink_hsh|
sink_hsh['name'] = -sink_hsh['name']
sink = DTAS::Sink.load(sink_hsh)
- sink.env = to_omap(sink.env)
@sinks[sink.name] = sink
end
end
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 3/9] dtas: drop unnecessary "require 'yaml'" statements
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
2022-01-23 21:37 ` [PATCH 1/9] deduplicate and freeze pathnames + metadata Eric Wong
2022-01-23 21:37 ` [PATCH 2/9] player: remove omap conversion Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 4/9] dtas-tl prune: cull missing files from tracklist Eric Wong
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
We use DTAS.yaml_load to wrap all YAML.*load calls, but
we still need "require 'yaml'" for various .to_yaml calls.
---
bin/dtas-console | 1 -
bin/dtas-msinkctl | 1 -
bin/dtas-player | 3 +--
bin/dtas-readahead | 1 -
bin/dtas-splitfx | 1 -
bin/dtas-tl | 1 -
lib/dtas/edit_client.rb | 1 -
lib/dtas/player.rb | 1 +
lib/dtas/sink.rb | 3 +--
lib/dtas/source/splitfx.rb | 1 -
test/player_integration.rb | 3 +--
test/test_encoding.rb | 1 -
test/test_splitfx.rb | 1 -
test/test_tfx.rb | 1 -
14 files changed, 4 insertions(+), 16 deletions(-)
diff --git a/bin/dtas-console b/bin/dtas-console
index a519ba9..eedd0f0 100755
--- a/bin/dtas-console
+++ b/bin/dtas-console
@@ -11,7 +11,6 @@
require 'dtas/process'
require 'dtas/format'
include DTAS::Process
-require 'yaml'
begin
require 'curses'
rescue LoadError
diff --git a/bin/dtas-msinkctl b/bin/dtas-msinkctl
index 6fb3863..79c7f26 100755
--- a/bin/dtas-msinkctl
+++ b/bin/dtas-msinkctl
@@ -2,7 +2,6 @@
# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
-require 'yaml'
require 'dtas/unix_client'
usage = "#$0 <active-set|active-add|active-sub|nonblock|active> SINK"
c = DTAS::UNIXClient.new
diff --git a/bin/dtas-player b/bin/dtas-player
index 21bc45f..c926e5f 100755
--- a/bin/dtas-player
+++ b/bin/dtas-player
@@ -1,9 +1,8 @@
#!/usr/bin/env ruby
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
Thread.abort_on_exception = $stderr.sync = $stdout.sync = true
-require 'yaml'
require 'dtas/player'
sock = (ENV["DTAS_PLAYER_SOCK"] || File.expand_path("~/.dtas/player.sock"))
state = (ENV["DTAS_PLAYER_STATE"] ||
diff --git a/bin/dtas-readahead b/bin/dtas-readahead
index 996142c..d3c6e79 100755
--- a/bin/dtas-readahead
+++ b/bin/dtas-readahead
@@ -12,7 +12,6 @@
@ffprobe = 'ffprobe'
@avprobe = 'avprobe'
-require 'yaml'
require 'io/wait'
require 'dtas/unix_client'
require 'dtas/process'
diff --git a/bin/dtas-splitfx b/bin/dtas-splitfx
index 05e71e5..6ce6521 100755
--- a/bin/dtas-splitfx
+++ b/bin/dtas-splitfx
@@ -2,7 +2,6 @@
# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
-require 'yaml'
require 'optparse'
require 'dtas/splitfx'
usage = "#$0 [-n|--dry-run][-j [JOBS]][-s|--silent] SPLITFX_FILE.yml [TARGET]"
diff --git a/bin/dtas-tl b/bin/dtas-tl
index c1af933..2b34937 100755
--- a/bin/dtas-tl
+++ b/bin/dtas-tl
@@ -23,7 +23,6 @@ def fix_enc!(str, enc)
def do_edit(c)
require 'dtas/edit_client'
- require 'yaml'
require 'tempfile'
extend DTAS::EditClient
tmp = Tempfile.new(%w(dtas-tl-edit .txt))
diff --git a/lib/dtas/edit_client.rb b/lib/dtas/edit_client.rb
index 036d036..2bdc4d8 100644
--- a/lib/dtas/edit_client.rb
+++ b/lib/dtas/edit_client.rb
@@ -2,7 +2,6 @@
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'tempfile'
-require 'yaml'
require_relative 'unix_client'
require_relative 'disclaimer'
diff --git a/lib/dtas/player.rb b/lib/dtas/player.rb
index 3db15c3..06ba788 100644
--- a/lib/dtas/player.rb
+++ b/lib/dtas/player.rb
@@ -2,6 +2,7 @@
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'shellwords'
+require 'yaml'
require_relative '../dtas'
require_relative 'xs'
require_relative 'source'
diff --git a/lib/dtas/sink.rb b/lib/dtas/sink.rb
index 735cdef..966bab4 100644
--- a/lib/dtas/sink.rb
+++ b/lib/dtas/sink.rb
@@ -1,7 +1,6 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
-require 'yaml'
require_relative '../dtas'
require_relative 'pipe'
require_relative 'process'
diff --git a/lib/dtas/source/splitfx.rb b/lib/dtas/source/splitfx.rb
index afeb6a3..2268404 100644
--- a/lib/dtas/source/splitfx.rb
+++ b/lib/dtas/source/splitfx.rb
@@ -1,7 +1,6 @@
# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
-require 'yaml'
require_relative 'sox'
require_relative '../splitfx'
require_relative '../watchable'
diff --git a/test/player_integration.rb b/test/player_integration.rb
index 4ad0e0c..66d1c6e 100644
--- a/test/player_integration.rb
+++ b/test/player_integration.rb
@@ -1,11 +1,10 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require './test/helper'
require 'dtas/player'
require 'dtas/state_file'
require 'dtas/unix_client'
-require 'yaml'
require 'tempfile'
require 'shellwords'
require 'timeout'
diff --git a/test/test_encoding.rb b/test/test_encoding.rb
index 666d185..5cd5da7 100644
--- a/test/test_encoding.rb
+++ b/test/test_encoding.rb
@@ -3,7 +3,6 @@
# frozen_string_literal: true
require './test/helper'
require 'dtas'
-require 'yaml'
class TestEncoding < Testcase
def test_encoding
diff --git a/test/test_splitfx.rb b/test/test_splitfx.rb
index e3bd19d..f2e0e09 100644
--- a/test/test_splitfx.rb
+++ b/test/test_splitfx.rb
@@ -1,7 +1,6 @@
# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
-require 'yaml'
require 'dtas/splitfx'
require 'thread'
require_relative 'helper'
diff --git a/test/test_tfx.rb b/test/test_tfx.rb
index 51b1900..be68079 100644
--- a/test/test_tfx.rb
+++ b/test/test_tfx.rb
@@ -4,7 +4,6 @@
require './test/helper'
require 'dtas/tfx'
require 'dtas/format'
-require 'yaml'
class TestTFX < Testcase
def rate
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 4/9] dtas-tl prune: cull missing files from tracklist
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
` (2 preceding siblings ...)
2022-01-23 21:37 ` [PATCH 3/9] dtas: drop unnecessary "require 'yaml'" statements Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 5/9] dtas-tl: drop encoding hacks, use binary stdout+stderr Eric Wong
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
This helps for folks creating and replacing many throwaway files
while editing with dtas-splitfx, or compulsive renamers.
---
Documentation/dtas-tl.pod | 4 ++-
bin/dtas-tl | 52 ++++++++++++++++++++++++---------------
2 files changed, 35 insertions(+), 21 deletions(-)
diff --git a/Documentation/dtas-tl.pod b/Documentation/dtas-tl.pod
index ac1b47d..b5a4b31 100644
--- a/Documentation/dtas-tl.pod
+++ b/Documentation/dtas-tl.pod
@@ -60,6 +60,8 @@ optionally seek to POS. POS should be a timestamp in HH:MM:SS.FRAC format.
=item prev - play the previous track in the tracklist
+=item prune - cull non-existent pathnames from the tracklist
+
=item repeat 1 - repeat the current track
=item repeat false - disable repeat
@@ -120,7 +122,7 @@ No subscription is necessary to post to the mailing list.
=head1 COPYRIGHT
-Copyright 2013-2020 all contributors L<mailto:dtas-all@nongnu.org>
+Copyright all contributors L<mailto:dtas-all@nongnu.org>
License: GPL-3.0+ L<https://www.gnu.org/licenses/gpl-3.0.txt>
diff --git a/bin/dtas-tl b/bin/dtas-tl
index 2b34937..767b02d 100755
--- a/bin/dtas-tl
+++ b/bin/dtas-tl
@@ -21,6 +21,18 @@ def fix_enc!(str, enc)
str.force_encoding(Encoding::ASCII_8BIT) unless str.valid_encoding?
end
+def each_track(c)
+ enc = Encoding.default_external
+ get_track_ids(c).each_slice(128) do |track_ids|
+ res = c.req("tl get #{track_ids.join(' ')}")
+ res = Shellwords.split(res.sub!(/\A\d+ /, ''))
+ while line = res.shift
+ fix_enc!(line, enc)
+ yield line
+ end
+ end
+end
+
def do_edit(c)
require 'dtas/edit_client'
require 'tempfile'
@@ -30,19 +42,13 @@ def do_edit(c)
tmp_path = tmp.path
orig = []
orig_idx = {}
- enc = Encoding.default_external
- get_track_ids(c).each_slice(128) do |track_ids|
- res = c.req("tl get #{track_ids.join(' ')}")
- res = Shellwords.split(res.sub!(/\A\d+ /, ''))
- while line = res.shift
- line.sub!(/\A(\d+)=/n, '') or abort "unexpected line=#{line.inspect}\n"
- fix_enc!(line, enc)
- track_id = $1.to_i
- orig_idx[track_id] = orig.size
- orig << track_id
- tmp.write("#{Shellwords.escape(line)} =#{track_id}\n")
- end
+ each_track(c) do |line|
+ line.sub!(/\A(\d+)=/n, '') or abort "unexpected line=#{line.inspect}\n"
+ track_id = $1.to_i
+ orig_idx[track_id] = orig.size
+ orig << track_id
+ tmp.write("#{Shellwords.escape(line)} =#{track_id}\n")
end
tmp.flush
@@ -139,15 +145,21 @@ def add_after(c, argv, last_id)
c = DTAS::UNIXClient.new
case cmd = ARGV[0]
-when "cat"
- enc = Encoding.default_external
- get_track_ids(c).each_slice(128) do |track_ids|
- res = c.req("tl get #{track_ids.join(' ')}")
- res = Shellwords.split(res.sub!(/\A\d+ /, ''))
- while line = res.shift
- fix_enc!(line, enc)
- print "#{line}\n"
+when 'cat'
+ each_track(c) { |line| print "#{line}\n" }
+when 'prune'
+ each_track(c) do |line|
+ line.sub!(/\A(\d+)=/n, '') or abort "unexpected line=#{line.inspect}\n"
+ track_id = $1.to_i
+ ok = false
+ begin
+ st = File.stat(line)
+ ok = st.readable? && st.size?
+ rescue Errno::ENOENT, Errno::ENOTDIR, Errno::EACCES => e
+ warn "# #{line}: #{e.class}"
+ # raise other exceptions
end
+ c.req("tl remove #{track_id}") unless ok
end
when 'aac' # add-after-current
ARGV.shift
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 5/9] dtas-tl: drop encoding hacks, use binary stdout+stderr
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
` (3 preceding siblings ...)
2022-01-23 21:37 ` [PATCH 4/9] dtas-tl prune: cull missing files from tracklist Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 6/9] use IO#wait_readable consistently Eric Wong
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
Try to consistently treat pathnames as binary blobs everywhere,
since POSIX FSes allow everything but "\0" in pathnames.
---
bin/dtas-tl | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/bin/dtas-tl b/bin/dtas-tl
index 767b02d..e58ee31 100755
--- a/bin/dtas-tl
+++ b/bin/dtas-tl
@@ -7,6 +7,8 @@
# itself is also unstable, but better than this one probably).
require 'dtas/unix_client'
require 'shellwords'
+$stdout.binmode
+$stderr.binmode
def get_track_ids(c)
track_ids = c.req("tl tracks")
@@ -16,18 +18,11 @@ def get_track_ids(c)
track_ids
end
-def fix_enc!(str, enc)
- str.force_encoding(enc)
- str.force_encoding(Encoding::ASCII_8BIT) unless str.valid_encoding?
-end
-
def each_track(c)
- enc = Encoding.default_external
get_track_ids(c).each_slice(128) do |track_ids|
res = c.req("tl get #{track_ids.join(' ')}")
res = Shellwords.split(res.sub!(/\A\d+ /, ''))
while line = res.shift
- fix_enc!(line, enc)
yield line
end
end
@@ -109,7 +104,6 @@ def do_edit(c)
path = File.expand_path(path)
unless File.exist?(path)
path = orig.dup
- fix_enc!(path, enc)
path = Shellwords.split(path)[0]
path = File.expand_path(path)
end
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 6/9] use IO#wait_readable consistently
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
` (4 preceding siblings ...)
2022-01-23 21:37 ` [PATCH 5/9] dtas-tl: drop encoding hacks, use binary stdout+stderr Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 7/9] get rid of DTAS::Nonblock wrapper for Ruby <= 2.0 Eric Wong
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
Since Ruby 2.3, it no longer checks FIONREAD, and we require
Ruby 2.3+ nowadays, so drop our IO.select-based workarounds.
---
bin/dtas-readahead | 16 ++--------------
lib/dtas/unix_client.rb | 4 ++--
2 files changed, 4 insertions(+), 16 deletions(-)
diff --git a/bin/dtas-readahead b/bin/dtas-readahead
index d3c6e79..75bfbe8 100755
--- a/bin/dtas-readahead
+++ b/bin/dtas-readahead
@@ -28,18 +28,6 @@
@redir = { err: null, out: null, in: null }.freeze
require 'pp'
-if RUBY_VERSION.to_r >= '2.3'.to_r
- # Old Rubies did FIONREAD, which breaks on SOCK_SEQPACKET
- def wait_read(w, timeout)
- w.to_io.wait_readable(timeout)
- end
-else
- def wait_read(w, timeout)
- r = IO.select([w], nil, nil, timeout)
- r ? r[0] : nil
- end
-end
-
def seek_to_cur_pos(cur_pid, fp)
cur_fd = []
fpst = fp.stat
@@ -120,7 +108,7 @@ def do_ra(fp, pos, w)
len -= n
# stop reading immediately if there's an event
- if wait_read(w, 0)
+ if w.to_io.wait_readable(0)
adj = @todo_ra
pos += size
break
@@ -220,7 +208,7 @@ def do_open(path)
fp.close if fp && !fp.closed?
fp = timeout = nil
end
- r = wait_read(w, timeout)
+ r = w.to_io.wait_readable(timeout)
p w.res_wait if r
rescue EOFError
abort "dtas-player exited"
diff --git a/lib/dtas/unix_client.rb b/lib/dtas/unix_client.rb
index 8aa953c..71f833c 100644
--- a/lib/dtas/unix_client.rb
+++ b/lib/dtas/unix_client.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require_relative '../dtas'
@@ -39,7 +39,7 @@ def req(args, timeout = nil)
end
def res_wait(timeout = nil)
- IO.select([@to_io], nil, nil, timeout)
+ @to_io.wait_readable(timeout)
nr = @to_io.nread
nr > 0 or raise EOFError, "unexpected EOF from server"
@to_io.recv(nr)
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 7/9] get rid of DTAS::Nonblock wrapper for Ruby <= 2.0
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
` (5 preceding siblings ...)
2022-01-23 21:37 ` [PATCH 6/9] use IO#wait_readable consistently Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 8/9] unix_accepted: drop Ruby < 2.3 support code Eric Wong
2022-01-23 21:37 ` [PATCH 9/9] do not check IO#closed? before calling IO#close Eric Wong
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
We require 2.3+, nowadays, so jettison a bunch of code.
---
lib/dtas/buffer/read_write.rb | 3 +--
lib/dtas/nonblock.rb | 24 ------------------------
lib/dtas/pipe.rb | 5 ++---
lib/dtas/process.rb | 5 ++---
lib/dtas/sigevent/fiddle_efd.rb | 3 +--
lib/dtas/sigevent/pipe.rb | 5 ++---
lib/dtas/unix_server.rb | 16 ++--------------
lib/dtas/watchable.rb | 3 +--
lib/dtas/watchable/fiddle_ino.rb | 2 +-
9 files changed, 12 insertions(+), 54 deletions(-)
delete mode 100644 lib/dtas/nonblock.rb
diff --git a/lib/dtas/buffer/read_write.rb b/lib/dtas/buffer/read_write.rb
index fdf820c..8fdb25d 100644
--- a/lib/dtas/buffer/read_write.rb
+++ b/lib/dtas/buffer/read_write.rb
@@ -1,10 +1,9 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'io/nonblock'
require_relative '../../dtas'
require_relative '../pipe'
-require_relative '../nonblock'
# compatibility code for non-Linux systems lacking "splice" support.
# Used only by -player
diff --git a/lib/dtas/nonblock.rb b/lib/dtas/nonblock.rb
deleted file mode 100644
index 2cf086c..0000000
--- a/lib/dtas/nonblock.rb
+++ /dev/null
@@ -1,24 +0,0 @@
-# Copyright (C) 2015-2020 all contributors <dtas-all@nongnu.org>
-# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
-
-class DTAS::Nonblock < IO # :nodoc:
- if RUBY_VERSION.to_f <= 2.0
- EX = {}.freeze
- def read_nonblock(len, buf = nil, opts = EX)
- super(len, buf)
- rescue IO::WaitReadable
- raise if opts[:exception]
- :wait_readable
- rescue EOFError
- raise if opts[:exception]
- nil
- end
-
- def write_nonblock(buf, opts = EX)
- super(buf)
- rescue IO::WaitWritable
- raise if opts[:exception]
- :wait_writable
- end
- end
-end
diff --git a/lib/dtas/pipe.rb b/lib/dtas/pipe.rb
index 34d50bd..a7b02b0 100644
--- a/lib/dtas/pipe.rb
+++ b/lib/dtas/pipe.rb
@@ -1,12 +1,11 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require_relative '../dtas'
require_relative 'writable_iter'
-require_relative 'nonblock'
# pipe wrapper for -player sinks
-class DTAS::Pipe < DTAS::Nonblock # :nodoc:
+class DTAS::Pipe < IO # :nodoc:
include DTAS::WritableIter
attr_accessor :sink
diff --git a/lib/dtas/process.rb b/lib/dtas/process.rb
index d00716f..02bf77e 100644
--- a/lib/dtas/process.rb
+++ b/lib/dtas/process.rb
@@ -5,7 +5,6 @@
require 'shellwords'
require_relative '../dtas'
require_relative 'xs'
-require_relative 'nonblock'
# process management helpers
module DTAS::Process # :nodoc:
@@ -88,12 +87,12 @@ def qx(env, cmd = {}, opts = {})
env = {}
end
buf = ''.b
- r, w = DTAS::Nonblock.pipe
+ r, w = IO.pipe
opts = opts.merge(out: w)
r.binmode
no_raise = opts.delete(:no_raise)
if err_str = opts.delete(:err_str)
- re, we = DTAS::Nonblock.pipe
+ re, we = IO.pipe
re.binmode
opts[:err] = we
end
diff --git a/lib/dtas/sigevent/fiddle_efd.rb b/lib/dtas/sigevent/fiddle_efd.rb
index ec9ec65..8bfa332 100644
--- a/lib/dtas/sigevent/fiddle_efd.rb
+++ b/lib/dtas/sigevent/fiddle_efd.rb
@@ -4,7 +4,6 @@
# used in various places for safe wakeups from IO.select via signals
# This requires a modern GNU/Linux system with eventfd(2) support
-require_relative '../nonblock'
require 'fiddle'
class DTAS::Sigevent # :nodoc:
@@ -18,7 +17,7 @@ class DTAS::Sigevent # :nodoc:
def initialize
fd = EventFD.call(0, 02000000|00004000) # EFD_CLOEXEC|EFD_NONBLOCK
raise "eventfd failed: #{Fiddle.last_error}" if fd < 0
- @to_io = DTAS::Nonblock.for_fd(fd)
+ @to_io = IO.for_fd(fd)
@buf = ''.b
end
diff --git a/lib/dtas/sigevent/pipe.rb b/lib/dtas/sigevent/pipe.rb
index e6fbbf2..6c3b83c 100644
--- a/lib/dtas/sigevent/pipe.rb
+++ b/lib/dtas/sigevent/pipe.rb
@@ -1,15 +1,14 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
# used in various places for safe wakeups from IO.select via signals
# A fallback for non-Linux systems lacking the "splice" syscall
-require_relative '../nonblock'
class DTAS::Sigevent # :nodoc:
attr_reader :to_io
def initialize
- @to_io, @wr = DTAS::Nonblock.pipe
+ @to_io, @wr = IO.pipe
@rbuf = ''.b
end
diff --git a/lib/dtas/unix_server.rb b/lib/dtas/unix_server.rb
index cad3fc4..60ab86c 100644
--- a/lib/dtas/unix_server.rb
+++ b/lib/dtas/unix_server.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'socket'
@@ -59,7 +59,7 @@ def write_failed(client, e)
def readable_iter
# we do not do anything with the block passed to us
- case rv = accept_nonblock
+ case rv = @to_io.accept_nonblock(exception: false)
when :wait_readable then return rv
else
@readers[DTAS::UNIXAccepted.new(rv[0])] = true
@@ -114,16 +114,4 @@ def run_once
wait_ctl(io, io.readable_iter { |_io, msg| yield(_io, msg) })
end
end
-
- if RUBY_VERSION.to_f >= 2.3
- def accept_nonblock
- @to_io.accept_nonblock(exception: false)
- end
- else
- def accept_nonblock
- @to_io.accept_nonblock
- rescue Errno::EAGAIN, Errno::ECONNABORTED, Errno::EPROTO
- :wait_readable
- end
- end
end
diff --git a/lib/dtas/watchable.rb b/lib/dtas/watchable.rb
index 6168bf3..445bf98 100644
--- a/lib/dtas/watchable.rb
+++ b/lib/dtas/watchable.rb
@@ -1,8 +1,7 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require_relative '../dtas'
-require_relative 'nonblock'
begin
module DTAS::Watchable # :nodoc:
module InotifyCommon # :nodoc:
diff --git a/lib/dtas/watchable/fiddle_ino.rb b/lib/dtas/watchable/fiddle_ino.rb
index b24bbf6..3ec72a1 100644
--- a/lib/dtas/watchable/fiddle_ino.rb
+++ b/lib/dtas/watchable/fiddle_ino.rb
@@ -22,7 +22,7 @@ class DTAS::Watchable::InotifyReadableIter # :nodoc:
def initialize # :nodoc:
fd = Inotify_init.call(02000000 | 04000) # CLOEXEC | NONBLOCK
raise "inotify_init failed: #{Fiddle.last_error}" if fd < 0
- @to_io = DTAS::Nonblock.for_fd(fd)
+ @to_io = IO.for_fd(fd)
@buf = ''.b
@q = []
end
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 8/9] unix_accepted: drop Ruby < 2.3 support code
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
` (6 preceding siblings ...)
2022-01-23 21:37 ` [PATCH 7/9] get rid of DTAS::Nonblock wrapper for Ruby <= 2.0 Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
2022-01-23 21:37 ` [PATCH 9/9] do not check IO#closed? before calling IO#close Eric Wong
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
This should save a little bit of memory for current users.
---
lib/dtas/unix_accepted.rb | 49 ++++++++++-----------------------------
1 file changed, 12 insertions(+), 37 deletions(-)
diff --git a/lib/dtas/unix_accepted.rb b/lib/dtas/unix_accepted.rb
index ec7f3ef..a84eade 100644
--- a/lib/dtas/unix_accepted.rb
+++ b/lib/dtas/unix_accepted.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'socket'
@@ -10,23 +10,22 @@ class DTAS::UNIXAccepted # :nodoc:
def initialize(sock)
@to_io = sock
- @send_buf = []
+ @sbuf = []
end
# public API (for DTAS::Player)
# returns :wait_readable on success
def emit(msg)
- buffered = @send_buf.size
- if buffered == 0
- case rv = sendmsg_nonblock(msg)
+ if @sbuf.empty?
+ case rv = @to_io.sendmsg_nonblock(msg, Socket::MSG_EOR, exception: false)
when :wait_writable
- @send_buf << msg
+ @sbuf << msg
rv
else
:wait_readable
end
- else # buffered > 0
- @send_buf << msg
+ else
+ @sbuf << msg
:wait_writable
end
rescue => e
@@ -35,11 +34,11 @@ def emit(msg)
# flushes pending data if it got buffered
def writable_iter
- case sendmsg_nonblock(@send_buf[0])
+ case @to_io.sendmsg_nonblock(@sbuf[0], Socket::MSG_EOR, exception: false)
when :wait_writable then return :wait_writable
else
- @send_buf.shift
- @send_buf.empty? ? :wait_readable : :wait_writable
+ @sbuf.shift
+ @sbuf.empty? ? :wait_readable : :wait_writable
end
rescue => e
e
@@ -51,13 +50,13 @@ def readable_iter
# EOF, assume no spurious wakeups for SOCK_SEQPACKET
return nil if nread == 0
- case msg = recv_nonblock(nread)
+ case msg = @to_io.recv_nonblock(nread, exception: false)
when :wait_readable then return msg
when '', nil then return nil # EOF
else
yield(self, msg) # DTAS::Player deals with this
end
- @send_buf.empty? ? :wait_readable : :wait_writable
+ @sbuf.empty? ? :wait_readable : :wait_writable
rescue SystemCallError
nil
end
@@ -69,28 +68,4 @@ def close
def closed?
@to_io.closed?
end
-
- if RUBY_VERSION.to_f >= 2.3
- def sendmsg_nonblock(msg)
- @to_io.sendmsg_nonblock(msg, Socket::MSG_EOR, exception: false)
- end
-
- def recv_nonblock(len)
- @to_io.recv_nonblock(len, exception: false)
- end
- else
- def sendmsg_nonblock(msg)
- @to_io.sendmsg_nonblock(msg, Socket::MSG_EOR)
- rescue IO::WaitWritable
- :wait_writable
- end
-
- def recv_nonblock(len)
- @to_io.recv_nonblock(len)
- rescue IO::WaitReadable
- :wait_readable
- rescue EOFError
- nil
- end
- end
end
^ permalink raw reply related [flat|nested] 10+ messages in thread
* [PATCH 9/9] do not check IO#closed? before calling IO#close
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
` (7 preceding siblings ...)
2022-01-23 21:37 ` [PATCH 8/9] unix_accepted: drop Ruby < 2.3 support code Eric Wong
@ 2022-01-23 21:37 ` Eric Wong
8 siblings, 0 replies; 10+ messages in thread
From: Eric Wong @ 2022-01-23 21:37 UTC (permalink / raw)
To: dtas-all
IO#close is idempotent since Ruby 2.3, so reduce our instruction
footprint to save some memory.
---
bin/dtas-readahead | 2 +-
lib/dtas/buffer.rb | 4 ++--
test/test_buffer.rb | 2 +-
test/test_unixserver.rb | 2 +-
4 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/bin/dtas-readahead b/bin/dtas-readahead
index 75bfbe8..6da5f88 100755
--- a/bin/dtas-readahead
+++ b/bin/dtas-readahead
@@ -205,7 +205,7 @@ def do_open(path)
timeout = 0 if timeout < 0
else
work.each_value(&:close).clear
- fp.close if fp && !fp.closed?
+ fp.close if fp
fp = timeout = nil
end
r = w.to_io.wait_readable(timeout)
diff --git a/lib/dtas/buffer.rb b/lib/dtas/buffer.rb
index 54487c5..0688af9 100644
--- a/lib/dtas/buffer.rb
+++ b/lib/dtas/buffer.rb
@@ -1,4 +1,4 @@
-# Copyright (C) 2013-2020 all contributors <dtas-all@nongnu.org>
+# Copyright (C) all contributors <dtas-all@nongnu.org>
# License: GPL-3.0+ <https://www.gnu.org/licenses/gpl-3.0.txt>
# frozen_string_literal: true
require 'io/wait'
@@ -45,7 +45,7 @@ def to_hsh
def __dst_error(dst, e)
warn "dropping #{dst.inspect} due to error: #{e.message} (#{e.class})"
- dst.close unless dst.closed?
+ dst.close
end
# This will modify targets
diff --git a/test/test_buffer.rb b/test/test_buffer.rb
index b89a090..a47e2d4 100644
--- a/test/test_buffer.rb
+++ b/test/test_buffer.rb
@@ -11,7 +11,7 @@ class TestBuffer < Testcase
@@max_size = nil if @@max_size == 0
def teardown
- @to_close.each { |io| io.close unless io.closed? }
+ @to_close.each(&:close)
end
def setup
diff --git a/test/test_unixserver.rb b/test/test_unixserver.rb
index b061af0..7e99b9e 100644
--- a/test/test_unixserver.rb
+++ b/test/test_unixserver.rb
@@ -21,7 +21,7 @@ def test_close
end
def teardown
- @clients.each { |io| io.close unless io.closed? }
+ @clients.each(&:close)
if File.exist?(@tmp.path)
@tmp.close!
else
^ permalink raw reply related [flat|nested] 10+ messages in thread
end of thread, other threads:[~2022-01-23 21:38 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-01-23 21:37 [PATCH 0/9] various cuts for memory savings Eric Wong
2022-01-23 21:37 ` [PATCH 1/9] deduplicate and freeze pathnames + metadata Eric Wong
2022-01-23 21:37 ` [PATCH 2/9] player: remove omap conversion Eric Wong
2022-01-23 21:37 ` [PATCH 3/9] dtas: drop unnecessary "require 'yaml'" statements Eric Wong
2022-01-23 21:37 ` [PATCH 4/9] dtas-tl prune: cull missing files from tracklist Eric Wong
2022-01-23 21:37 ` [PATCH 5/9] dtas-tl: drop encoding hacks, use binary stdout+stderr Eric Wong
2022-01-23 21:37 ` [PATCH 6/9] use IO#wait_readable consistently Eric Wong
2022-01-23 21:37 ` [PATCH 7/9] get rid of DTAS::Nonblock wrapper for Ruby <= 2.0 Eric Wong
2022-01-23 21:37 ` [PATCH 8/9] unix_accepted: drop Ruby < 2.3 support code Eric Wong
2022-01-23 21:37 ` [PATCH 9/9] do not check IO#closed? before calling IO#close Eric Wong
Code repositories for project(s) associated with this public inbox
https://80x24.org/dtas.git/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for read-only IMAP folder(s) and NNTP newsgroup(s).