Piping two shell commands together

I want to pipe two system commands together from within ruby. cmd1 and cmd2 are arrays, e.g.

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

I think I can do the following,

IO.popen('-','r') do |p1|
   exec cmd1 unless p1
   IO.popen('-','w') do |p2|
     exec cmd2 unless p2
     p2.write(p1.read)
   end
end

but I don't like the line "p2.write(p1.read)", because I think it will read into memory from p1 before turning around and writing it to p2. This may work for the above example, but it's not very pipelike and I worry that read might return early (I'm not sure what the semantics of that are in Ruby, but in C I'd have to account for that).

Plus I don't like the popen '-'/exec combination. I'd love a more elgant, efficient, and/or safer idiom if anyone can think of one.

`#{cmd1.join(' ')} | #{cmd2.join(' ')}`

Those are backticks. Let the system handle piping with the '|' character.

Jason

···

On 8/2/07, Hans Fugal <fugalh@zianet.com> wrote:

I want to pipe two system commands together from within ruby. cmd1 and
cmd2 are arrays, e.g.

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

I think I can do the following,

IO.popen('-','r') do |p1|
   exec cmd1 unless p1
   IO.popen('-','w') do |p2|
     exec cmd2 unless p2
     p2.write(p1.read)
   end
end

but I don't like the line "p2.write(p1.read)", because I think it will
read into memory from p1 before turning around and writing it to p2.
This may work for the above example, but it's not very pipelike and I
worry that read might return early (I'm not sure what the semantics of
that are in Ruby, but in C I'd have to account for that).

Plus I don't like the popen '-'/exec combination. I'd love a more
elgant, efficient, and/or safer idiom if anyone can think of one.

I don't know about the arrays, but you can say IO.popen ("oggdec |
lame").... m.

···

Hans Fugal <fugalh@zianet.com> wrote:

I want to pipe two system commands together from within ruby. cmd1 and
cmd2 are arrays, e.g.

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

--
matt neuburg, phd = matt@tidbits.com, Matt Neuburg’s Home Page
Tiger - http://www.takecontrolbooks.com/tiger-customizing.html
AppleScript - http://www.amazon.com/gp/product/0596102119
Read TidBITS! It's free and smart. http://www.tidbits.com

an alternative:

cfp:~ > ruby a.rb
1073741824

cfp:~ > cat a.rb
require 'open4'

cmd1 = 'ruby -e" (2 ** 10).times{ STDOUT.write 0.chr * 2**20 } "'
cmd2 = 'ruby -e" puts ARGF.read.size "'

stdin = IO.popen cmd1

stdout, stderr = '', ''

status = Open4.spawn cmd2, 0=>stdin, 1=>stdout, 2=>stderr

puts stdout
puts 2 ** 30

a @ http://drawohara.com/

···

On Aug 2, 2007, at 10:40 AM, Hans Fugal wrote:

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

I think I can do the following,

IO.popen('-','r') do |p1|
  exec cmd1 unless p1
  IO.popen('-','w') do |p2|
    exec cmd2 unless p2
    p2.write(p1.read)
  end
end

--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

Hi,

At Fri, 3 Aug 2007 01:40:00 +0900,
Hans Fugal wrote in [ruby-talk:263040]:

I want to pipe two system commands together from within ruby. cmd1 and
cmd2 are arrays, e.g.

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

  IO.popen('-','r') do |p1|
    exec(*cmd1) unless p1
    IO.popen('-','w') do |p2|
      unless p2
        STDIN.reopen(p1)
        exec(*cmd2)
      end
    end
  end

···

--
Nobu Nakada

matt neuburg wrote:

···

Hans Fugal <fugalh@zianet.com> wrote:

I want to pipe two system commands together from within ruby. cmd1 and
cmd2 are arrays, e.g.

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

I don't know about the arrays, but you can say IO.popen ("oggdec |
lame").... m.

Yeah, that won't work here (unless there's a reliable way to shell-quote/escape oldpath and newpath).

I'm a little confused about oldPath. Once you have told IO.popen to
generate an IO instance (let's call it ios), whatever you write to ios
passes thru the process. So you don't actually need to tell IO.popen
about oldpath; you just write the data.

As for quoting, I just put quotes around it.

So, for example, a way to do LAME encoding with Ruby is like this:

oldpath = "/Users/mattneub/some sound file.aif"
newpath = "/Users/mattneub/some sound file.mp3"
s = "lame --preset standard - '#{newpath}'"
IO.popen(s, "r+") do |ios|
  ios.write(File.read(oldpath))
end

So maybe you could say:

s = "oggdec -R -o - - | lame ... - '#{newpath}'"

I've suggested -R because the oggdec man page warns not to write WAV to
stdout. But then I've omitted some lame params (...) because you'll
probably need to tell lame what kind of raw data it's receiving. I
haven't tried this, though.

m.

···

Hans Fugal <fugalh@zianet.com> wrote:

matt neuburg wrote:
> Hans Fugal <fugalh@zianet.com> wrote:
>
>> I want to pipe two system commands together from within ruby. cmd1 and
>> cmd2 are arrays, e.g.
>>
>> cmd1 = ['oggdec','-o','-',oldpath]
>> cmd2 = ['lame','-',newpath]
>
> I don't know about the arrays, but you can say IO.popen ("oggdec |
> lame").... m.
>

Yeah, that won't work here (unless there's a reliable way to
shell-quote/escape oldpath and newpath).

--
matt neuburg, phd = matt@tidbits.com, Matt Neuburg’s Home Page
Tiger - http://www.takecontrolbooks.com/tiger-customizing.html
AppleScript - http://www.amazon.com/gp/product/0596102119
Read TidBITS! It's free and smart. http://www.tidbits.com

Yes, I just tried it. (Had to download and compile and install the ogg
stuff first.) I started with a mono aiff file; then I ogg'ed using the
default setting. Then I said:

oldpath = "/Users/mattneub/some sound file.ogg"
newpath = "/Users/mattneub/some sound file.mp3"
s = "oggdec -R -o - - | lame -r -x -m m --preset standard -
'#{newpath}'"
IO.popen(s, "r+") do |ios|
  ios.write(File.read(oldpath))
end

Worked great. If your file isn't mono, leave out the "-m m". If your
native endianity does not require it, leave out the "-x". Looks pretty
"pipey" to me!

m.

···

matt neuburg <matt@tidbits.com> wrote:

So, for example, a way to do LAME encoding with Ruby is like this:

oldpath = "/Users/mattneub/some sound file.aif"
newpath = "/Users/mattneub/some sound file.mp3"
s = "lame --preset standard - '#{newpath}'"
IO.popen(s, "r+") do |ios|
  ios.write(File.read(oldpath))
end

So maybe you could say:

s = "oggdec -R -o - - | lame ... - '#{newpath}'"

I've suggested -R because the oggdec man page warns not to write WAV to
stdout. But then I've omitted some lame params (...) because you'll
probably need to tell lame what kind of raw data it's receiving. I
haven't tried this, though.

--
matt neuburg, phd = matt@tidbits.com, Matt Neuburg’s Home Page
Tiger - http://www.takecontrolbooks.com/tiger-customizing.html
AppleScript - http://www.amazon.com/gp/product/0596102119
Read TidBITS! It's free and smart. http://www.tidbits.com

matt neuburg wrote:

···

Hans Fugal <fugalh@zianet.com> wrote:

matt neuburg wrote:

Hans Fugal <fugalh@zianet.com> wrote:

I want to pipe two system commands together from within ruby. cmd1 and
cmd2 are arrays, e.g.

cmd1 = ['oggdec','-o','-',oldpath]
cmd2 = ['lame','-',newpath]

I don't know about the arrays, but you can say IO.popen ("oggdec |
lame").... m.

Yeah, that won't work here (unless there's a reliable way to shell-quote/escape oldpath and newpath).

I'm a little confused about oldPath. Once you have told IO.popen to
generate an IO instance (let's call it ios), whatever you write to ios
passes thru the process. So you don't actually need to tell IO.popen
about oldpath; you just write the data.

As for quoting, I just put quotes around it.

So, for example, a way to do LAME encoding with Ruby is like this:

oldpath = "/Users/mattneub/some sound file.aif"
newpath = "/Users/mattneub/some sound file.mp3"
s = "lame --preset standard - '#{newpath}'"
IO.popen(s, "r+") do |ios|
  ios.write(File.read(oldpath))
end

So maybe you could say:

s = "oggdec -R -o - - | lame ... - '#{newpath}'"

I've suggested -R because the oggdec man page warns not to write WAV to
stdout. But then I've omitted some lame params (...) because you'll
probably need to tell lame what kind of raw data it's receiving. I
haven't tried this, though.

What happens when you try to encode to newpath = "Beethoven's 5th" or something with an apostrophe?

I did come up with a working solution, which you can read about here: http://hans.fugal.net/blog/articles/2007/08/02/pipelining-processes-in-ruby

Cheers