Line wrapping in YAML output

Line wrapping in YAML output

[James Rowe]

[-- Attachment #1: Type: text/plain, Size: 561 bytes --]

Hi,

  I routinely find myself needing to use a “real language” when I want
to perform a quick hack with dtas purely to workaround the default line
wrapping in YAML output.  With unwrapped output sed/awk would often be
a viable solution from the shell prompt.

  Given that psych supports an options mapping where setting
``line_width: -1`` disables wrapping entirely, I’m wondering if there
would be support for disabling the line wrapping.  I’m also wondering if
I’m missing some drawbacks to doing so.

  Thoughts?

Thanks,

James

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Line wrapping in YAML output

[Eric Wong]

James Rowe <jnrowe@gmail.com> wrote:
> Hi,
> 
>   I routinely find myself needing to use a “real language” when I want
> to perform a quick hack with dtas purely to workaround the default line
> wrapping in YAML output.  With unwrapped output sed/awk would often be
> a viable solution from the shell prompt.

Understandable.  Do you use dtas-ctl or some other socket tool?

On the flip side, one of the reasons I picked YAML over JSON is
the indented + wrapped-by-default nature made it
$EDITOR-friendly.

In retrospect, I kinda wish I'd gone with JSON for a wire format
and maybe YAML or even RFC822 for $EDITOR-friendliness.

>   Given that psych supports an options mapping where setting
> ``line_width: -1`` disables wrapping entirely, I’m wondering if there
> would be support for disabling the line wrapping.  I’m also wondering if
> I’m missing some drawbacks to doing so.

My patch below seems to work.  The dtas-*edit tools had to be
updated for $EDITOR-friendliness.

dtas-ctl output could become difficult-to-read on the terminal;
maybe it could detect stdout is a terminal and rewrap in that
case.

------8<------
From: Eric Wong <e@yhbt.net>
Subject: [PATCH] player: disable line wrapping in YAML socket output

Introduce DTAS.yaml wrapper to consistently disable
line wrapping in player output.  To keep dtas-*edit
commands from having excessively long lines, we'll
re-wrap them before spawning text editors.
---
 bin/dtas-sinkedit                 |  1 +
 bin/dtas-sourceedit               |  1 +
 lib/dtas.rb                       |  4 ++++
 lib/dtas/player.rb                |  2 +-
 lib/dtas/player/client_handler.rb | 16 ++++++++--------
 lib/dtas/state_file.rb            |  2 +-
 6 files changed, 16 insertions(+), 10 deletions(-)

diff --git a/bin/dtas-sinkedit b/bin/dtas-sinkedit
index 3393aac..16ece0b 100755
--- a/bin/dtas-sinkedit
+++ b/bin/dtas-sinkedit
@@ -70,6 +70,7 @@
 else
   include DTAS::SpawnFix
   tmp = tmpyaml
+  buf = orig.to_yaml # re-enable line wrapping
   tmp_path = tmp.path
   do_update = lambda { commit_update.call(File.read(tmp_path)) }
   tmp.write(buf << DTAS_DISCLAIMER)
diff --git a/bin/dtas-sourceedit b/bin/dtas-sourceedit
index 23362c2..2e9dd57 100755
--- a/bin/dtas-sourceedit
+++ b/bin/dtas-sourceedit
@@ -57,6 +57,7 @@
 else
   include DTAS::SpawnFix
   tmp = tmpyaml
+  buf = orig.to_yaml # re-enable line wrapping
   tmp_path = tmp.path
   do_update = lambda { commit_update.call(File.read(tmp_path)) }
   tmp.write(buf << DTAS_DISCLAIMER)
diff --git a/lib/dtas.rb b/lib/dtas.rb
index 1eac704..23d5ea4 100644
--- a/lib/dtas.rb
+++ b/lib/dtas.rb
@@ -33,6 +33,10 @@ def self.libc
     end
   end
 
+  def self.yaml(obj)
+    obj.to_yaml(line_width: -1) # disable wrapping
+  end
+
   # String#-@ will deduplicate strings when Ruby 2.5 is released (Dec 2017)
   # https://bugs.ruby-lang.org/issues/13077
   if RUBY_VERSION.to_f >= 2.5
diff --git a/lib/dtas/player.rb b/lib/dtas/player.rb
index b39a2e7..f21b6a0 100644
--- a/lib/dtas/player.rb
+++ b/lib/dtas/player.rb
@@ -228,7 +228,7 @@ def dpc_clear(io, msg)
   end
 
   def dpc_queue(io, msg)
-    'cat' == msg[0] and io.emit(@queue.to_yaml)
+    'cat' == msg[0] and io.emit(DTAS.yaml(@queue))
   end
 
   def dpc_watch(io, _)
diff --git a/lib/dtas/player/client_handler.rb b/lib/dtas/player/client_handler.rb
index cf5442d..c11ed69 100644
--- a/lib/dtas/player/client_handler.rb
+++ b/lib/dtas/player/client_handler.rb
@@ -176,7 +176,7 @@ def dpc_sink(io, msg)
       io.emit("OK")
     when "cat"
       sink = @sinks[name] or return io.emit("ERR #{name} not found")
-      io.emit(sink.to_hash.to_yaml)
+      io.emit(DTAS.yaml(sink.to_hash))
     else
       io.emit("ERR unknown sink op #{msg[0]}")
     end
@@ -259,7 +259,7 @@ def current_expect_samples(in_samples) # @current.samples
   end
 
   def dpc_rg(io, msg)
-    return io.emit(@rg.to_hsh.to_yaml) if msg.empty?
+    return io.emit(DTAS.yaml(@rg.to_hsh)) if msg.empty?
     before = @rg.to_hsh
     msg.each do |kv|
       k, v = kv.split('=', 2)
@@ -333,7 +333,7 @@ def dpc_current(io, msg)
       end
     end
     tmp['tracklist'] = @tl.to_hsh(false)
-    io.emit(tmp.to_yaml)
+    io.emit(DTAS.yaml(tmp))
   end
 
   def __buf_reset(buf) # buf is always @sink_buf for now
@@ -351,12 +351,12 @@ def dpc_skip(io, msg)
   def play_pause_handler(io, command)
     prev = @paused
     __send__("do_#{command}")
-    io.emit({
+    io.emit(DTAS.yaml({
       "paused" => {
         "before" => prev,
         "after" => @paused,
       }
-    }.to_yaml)
+    }))
   end
 
   def do_pause
@@ -515,7 +515,7 @@ def dpc_source(io, msg)
     src = map[name] or return io.emit("ERR non-existent source name")
     case op
     when "cat"
-      io.emit(src.to_source_cat.to_yaml)
+      io.emit(DTAS.yaml(src.to_source_cat))
     when "ed"
       before = src.to_state_hash.inspect
       sd = src.source_defaults
@@ -642,7 +642,7 @@ def _dpc_tl_shuffle(io, msg)
     prev = (!!@tl.shuffle).to_s
     v = msg.shift
     case v
-    when 'debug' then return io.emit(@tl.shuffle.to_yaml) # TODO: remove
+    when 'debug' then return io.emit(DTAS.yaml(@tl.shuffle)) # TODO: remove
     when nil
     else
       set_bool(io, 'tl shuffle', v) do |b|
@@ -844,7 +844,7 @@ def dpc_cue(io, msg)
       case msg[0]
       when nil
         tmp = { "infile" => cur.infile, "cue" => bp.map(&:to_hash) }
-        io.emit(tmp.to_yaml)
+        io.emit(DTAS.yaml(tmp))
       when "next", "prev"
         return __bp_prev_next(io, msg, cur, bp)
       when 'seek' then return __bp_seek(io, msg, cur, bp)
diff --git a/lib/dtas/state_file.rb b/lib/dtas/state_file.rb
index eac3e2f..8d7d413 100644
--- a/lib/dtas/state_file.rb
+++ b/lib/dtas/state_file.rb
@@ -18,7 +18,7 @@ def tryload
   end
 
   def dump(obj, force_fsync = false)
-    yaml = obj.to_hsh.to_yaml.b
+    yaml = DTAS.yaml(obj.to_hsh).b
 
     # do not replace existing state file if there are no changes
     # this will be racy if we ever do async dumps or shared state

Re: Line wrapping in YAML output

[James Rowe]

[-- Attachment #1: Type: text/plain, Size: 2197 bytes --]

* Eric Wong (e@80x24.org) wrote:
> James Rowe <jnrowe@gmail.com> wrote:
> >   I routinely find myself needing to use a “real language” when I want
> > to perform a quick hack with dtas purely to workaround the default line
> > wrapping in YAML output.  With unwrapped output sed/awk would often be
> > a viable solution from the shell prompt.
> 
> Understandable.  Do you use dtas-ctl or some other socket tool?

  Pretty much always parsing dtas-ctl, because I hardly ever seem to
have a {nc,socat}-style tool with SOCK_SEQPACKET support.

> On the flip side, one of the reasons I picked YAML over JSON is
> the indented + wrapped-by-default nature made it
> $EDITOR-friendly.

  I hadn’t thought it through all that much, which is why I was asking
about possible drawbacks.

  You’ve pushed me enough to think that dropping a yaml2json¹ script in
~/bin would often be enough for many of my use cases.  That simple
change would allow fancy jid²/jq³ support along with hacky sed/awk
scripts.

> >   Given that psych supports an options mapping where setting
> > ``line_width: -1`` disables wrapping entirely, I’m wondering if there
> > would be support for disabling the line wrapping.  I’m also wondering if
> > I’m missing some drawbacks to doing so.
> 
> My patch below seems to work.  The dtas-*edit tools had to be
> updated for $EDITOR-friendliness.

  Oh wow, thanks!  WFM, but I’ll add that I didn’t expect you to do the
work ;)

> dtas-ctl output could become difficult-to-read on the terminal;
> maybe it could detect stdout is a terminal and rewrap in that
> case.

  I’m not really a fan of fiddling with behaviour on isatty(), as it
quietly changes behaviour when you’re perhaps not expecting it.  Lots of
tools do it though, so perhaps it is just me.

  So… I’m not unsure whether I still want this change.  There are
workarounds with few drawbacks, and nobody else appears to have
complained in the previous seven years about this.

Thanks,

James

1. ``puts JSON.pretty_generate(YAML.load(STDIN.read))`` in the laziest
   case.
2. https://github.com/simeji/jid/
3. https://github.com/stedolan/jq/

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Line wrapping in YAML output

[Eric Wong]

James Rowe <jnrowe@gmail.com> wrote:
> * Eric Wong (e@80x24.org) wrote:
> > James Rowe <jnrowe@gmail.com> wrote:
> > >   I routinely find myself needing to use a “real language” when I want
> > > to perform a quick hack with dtas purely to workaround the default line
> > > wrapping in YAML output.  With unwrapped output sed/awk would often be
> > > a viable solution from the shell prompt.
> > 
> > Understandable.  Do you use dtas-ctl or some other socket tool?
> 
>   Pretty much always parsing dtas-ctl, because I hardly ever seem to
> have a {nc,socat}-style tool with SOCK_SEQPACKET support.

It's not well-documented, but socat supports setting type= to
the numeric value of SOCK_SEQPACKET.  At least on Linux,
SOCK_SEQPACKET is 5 (ruby -rsocket -e 'puts Socket::SOCK_SEQPACKET'):

  echo current | socat UNIX-CONNECT:$HOME/.dtas/player.sock,type=5 -

> > On the flip side, one of the reasons I picked YAML over JSON is
> > the indented + wrapped-by-default nature made it
> > $EDITOR-friendly.
> 
>   I hadn’t thought it through all that much, which is why I was asking
> about possible drawbacks.
> 
>   You’ve pushed me enough to think that dropping a yaml2json¹ script in
> ~/bin would often be enough for many of my use cases.  That simple
> change would allow fancy jid²/jq³ support along with hacky sed/awk
> scripts.

Cool.  Fwiw, I also just found "yq" looking for a jq-like thing
for YAML: https://kislyuk.github.io/yq/

I haven't tried yq, and have no plans to: I prefer to avoid
software unless it's been in Debian for a while.

> > >   Given that psych supports an options mapping where setting
> > > ``line_width: -1`` disables wrapping entirely, I’m wondering if there
> > > would be support for disabling the line wrapping.  I’m also wondering if
> > > I’m missing some drawbacks to doing so.
> > 
> > My patch below seems to work.  The dtas-*edit tools had to be
> > updated for $EDITOR-friendliness.
> 
>   Oh wow, thanks!  WFM, but I’ll add that I didn’t expect you to do the
> work ;)

No problem,  I was curious how long much effort it took and
it wasn't much, at all.

> > dtas-ctl output could become difficult-to-read on the terminal;
> > maybe it could detect stdout is a terminal and rewrap in that
> > case.
> 
>   I’m not really a fan of fiddling with behaviour on isatty(), as it
> quietly changes behaviour when you’re perhaps not expecting it.  Lots of
> tools do it though, so perhaps it is just me.

Yeah, it's a bit surprising sometimes.  I often run commands in my
$EDITOR buffer and it wouldn't help that case.

>   So… I’m not unsure whether I still want this change.  There are

"not unsure"?  Based on the rest of what you've written,
I think you meant just to write either "unsure" or "not sure"?

> workarounds with few drawbacks, and nobody else appears to have
> complained in the previous seven years about this.

AFAIK, you're maybe the 3rd user of this? :)

I'm not exactly good at promoting software and have no plans
to break out of my introvert comfort zone  :>

Anyways, I'm not inclined to change this, either; since
reformatting YAML or converting to JSON is pretty easy
in Ruby/Perl/Python.

Re: Line wrapping in YAML output

[James Rowe]

* Eric Wong (e@80x24.org) wrote:
> James Rowe <jnrowe@gmail.com> wrote:
> >   Pretty much always parsing dtas-ctl, because I hardly ever seem to
> > have a {nc,socat}-style tool with SOCK_SEQPACKET support.
> 
> It's not well-documented, but socat supports setting type= to
> the numeric value of SOCK_SEQPACKET.  At least on Linux,
> SOCK_SEQPACKET is 5 (ruby -rsocket -e 'puts Socket::SOCK_SEQPACKET'):
> 
>   echo current | socat UNIX-CONNECT:$HOME/.dtas/player.sock,type=5 -

  Ah, thanks!  I hadn’t realised that.

> >   You’ve pushed me enough to think that dropping a yaml2json¹ script in
> > ~/bin would often be enough for many of my use cases.  That simple
> > change would allow fancy jid²/jq³ support along with hacky sed/awk
> > scripts.
> 
> Cool.  Fwiw, I also just found "yq" looking for a jq-like thing
> for YAML: https://kislyuk.github.io/yq/

  Looks like it is mostly just a more thought out implementation of my
suggestion¹.

> >   So… I’m not unsure whether I still want this change.  There are
> 
> "not unsure"?  Based on the rest of what you've written,
> I think you meant just to write either "unsure" or "not sure"?

  Oops, my bad.  I did mean “not sure”.

> > workarounds with few drawbacks, and nobody else appears to have
> > complained in the previous seven years about this.
> 
> AFAIK, you're maybe the 3rd user of this? :)

  In the Old World before The Collapse, I’d pushed dtas at $WORK.  There
were quite a few people happily using it, albeit mostly just to enqueue
a track or toggle their personal sinks on and off.  Perhaps, in time, we
will bring the count back up!

> Anyways, I'm not inclined to change this, either; since
> reformatting YAML or converting to JSON is pretty easy
> in Ruby/Perl/Python.

  Agree.

Thanks,

James

1. https://github.com/kislyuk/yq/blob/master/yq/__init__.py#L154

Re: Line wrapping in YAML output

[Eric Wong]

James Rowe <jnrowe@gmail.com> wrote:
> * Eric Wong (e@80x24.org) wrote:
> > AFAIK, you're maybe the 3rd user of this? :)
> 
>   In the Old World before The Collapse, I’d pushed dtas at $WORK.  There
> were quite a few people happily using it, albeit mostly just to enqueue
> a track or toggle their personal sinks on and off.  Perhaps, in time, we
> will bring the count back up!

Cool, good to know.  Btw, would you have any thoughts on whether
the current implementation language being Ruby has a positive or
negative effect on adoption?  Or if Perl might be more appealing
as it's already on more systems and has faster startup.

I've been considering migrating this to Perl, but the Perl 7
announcement has me worried it'll follow in making the same
mistakes as Ruby and Python w.r.t. breaking existing code.

That said, I've been thinking about a RCU-aware scripting
language with a stability promise similar to the Linux kernel
for a while, now (not that RCU is particular appropriate for
dtas, but who knows? :>).

Re: Line wrapping in YAML output

[James Rowe]

[Summer break…]

> > * Eric Wong (e@80x24.org) wrote:
> Btw, would you have any thoughts on whether the current implementation
> language being Ruby has a positive or negative effect on adoption?  Or
> if Perl might be more appealing as it's already on more systems and
> has faster startup.

  I believe the single biggest thing that could improve adoption would
be a Debian/Fedora package, preferably an official one.  I know the lack
of a system package was an annoyance for a few people I’ve spoken to
about dtas.  Needing to care about implementation language *somewhat*
disappears when an install is just a apt or dnf call, and the
integration extras such as service management makes life nicer.

  I get that packaging is a lot of work(and not always possible via
official channels), but I’m just putting it out there.

Thanks,

James

Re: Line wrapping in YAML output

[Eric Wong]

James Rowe <jnrowe@gmail.com> wrote:
> [Summer break…]
> 
> > > * Eric Wong (e@80x24.org) wrote:
> > Btw, would you have any thoughts on whether the current implementation
> > language being Ruby has a positive or negative effect on adoption?  Or
> > if Perl might be more appealing as it's already on more systems and
> > has faster startup.

On a side note, there may be Ruby start-time improvements next year.

>   I believe the single biggest thing that could improve adoption would
> be a Debian/Fedora package, preferably an official one.  I know the lack
> of a system package was an annoyance for a few people I’ve spoken to
> about dtas.  Needing to care about implementation language *somewhat*
> disappears when an install is just a apt or dnf call, and the
> integration extras such as service management makes life nicer.

I totally agree, and that's why I've tried to reduce
dependencies on unpackaged gems to reduce overhead for potential
packagers.  I generally avoid software which isn't in distros.

>   I get that packaging is a lot of work(and not always possible via
> official channels), but I’m just putting it out there.

Yes, it would be much work (definitely too much for me).  I know
somebody on the ruby-talk list was interested years ago, but I
forgot what came of it.  I guess we'll have to wait for someone
else to come along.

Fwiw, I also find 99% of marketing annoying and pushy; so there
won't be more of that aside from release announcements on ruby-talk.