IO.popen on Windows to filter stdout

ruby-talk
1

Hello all,

I've been googling and playing with this for a while, but I'm stuck
and I figured I'd ask in case anyone's done this before.

I'd like to use IO.popen, or something similar, to 'filter' the spew
coming from another programs stdout. I think I understand that I can
capture the stdout stream returned by IO.popen by using .readlines,
etc... But it still seems that the sub process is writing the output
directly to $stdout...

Of course, this is on windows, so I don't have proper fork, etc...

Is there a way to somehow "swallow" the stdout from the sub process so
it doesn't make it directly to the ruby process' stdout?

Thanks in advance for any insight,
-Harold

2

probably the other process is writing to stderr. try

   IO.popen( 'cmd 2>&1' ) do |pipe|
     pipe.each do |line|
       p line
     end
   end

alternatively, you may want to use systemu

   http://codeforpeople.com/lib/ruby/systemu

it's __very__ easy to block a process using IO.popen on windows.

regards.

-a

···

On Thu, 22 Feb 2007, Harold Hausman wrote:

Hello all,

I've been googling and playing with this for a while, but I'm stuck and I
figured I'd ask in case anyone's done this before.

I'd like to use IO.popen, or something similar, to 'filter' the spew coming
from another programs stdout. I think I understand that I can capture the
stdout stream returned by IO.popen by using .readlines, etc... But it still
seems that the sub process is writing the output directly to $stdout...

Of course, this is on windows, so I don't have proper fork, etc...

Is there a way to somehow "swallow" the stdout from the sub process so it
doesn't make it directly to the ruby process' stdout?

Thanks in advance for any insight,
-Harold

--
be kind whenever possible... it is always possible.
- the dalai lama

3

Hi Ara, thanks for taking the time to try and help me. What you're
saying about stderr here sounds right to me, but I tried what you have
above, and I'm still seeing the output I'm trying to 'squash' (read:
block) ... Here is my code:

···

On 2/22/07, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Thu, 22 Feb 2007, Harold Hausman wrote:

> <snip...>
>
> Of course, this is on windows, so I don't have proper fork, etc...
>
> Is there a way to somehow "swallow" the stdout from the sub process so it
> doesn't make it directly to the ruby process' stdout?

probably the other process is writing to stderr. try

   IO.popen( 'cmd 2>&1' ) do |pipe|
     pipe.each do |line|
       p line
     end
   end

#########################
puts "ruby-link in the house!"
the_cmd = "old-Link.exe "
ARGV.each do |arg|
  the_cmd << "\"#{arg}\" "
end

the_cmd << "2>&1"

puts "Calling: #{the_cmd}"

IO.popen( the_cmd )
#########################

And here is the output that I see: (apologies for badly wrapped lines)
ruby-link in the house!
Calling: old-Link.exe "@C:\long\path\to\some.rsp" "/NOLOGO"
"/ERRORREPORT:PROMPT" 2>&1
OLD-LINK :
<snip about 200k of nasty output from old-Link.exe>

------
So yeah, the output from old-Link.exe is still being filtered up to
the top and seen. What I'm looking for is a way to shut it up... :wink:

alternatively, you may want to use systemu

   http://codeforpeople.com/lib/ruby/systemu

I looked at systemu and it's not clear how it would help with this
problem. :expressionless: Apologies if I'm being dense.

it's __very__ easy to block a process using IO.popen on windows.

I thought so too. :wink:

Thanks again for your time,
-Harold

4

Hi Ara, thanks for taking the time to try and help me. What you're
saying about stderr here sounds right to me, but I tried what you have
above, and I'm still seeing the output I'm trying to 'squash' (read:
block) ... Here is my code:
#########################
puts "ruby-link in the house!"
the_cmd = "old-Link.exe "
ARGV.each do |arg|
  the_cmd << "\"#{arg}\" "
end

the_cmd << "2>&1"

puts "Calling: #{the_cmd}"

IO.popen( the_cmd )
#########################

And here is the output that I see: (apologies for badly wrapped lines)
ruby-link in the house!
Calling: old-Link.exe "@C:\long\path\to\some.rsp" "/NOLOGO"
"/ERRORREPORT:PROMPT" 2>&1
OLD-LINK :
<snip about 200k of nasty output from old-Link.exe>

hrrrm. this is a windozing type thing, but is it perhaps manipulating the
console directly? for instance, in unix, this can happen.

it's hard to say without seeing your code, and the old code, but you could try
this to prove to yourself that stdout and stderr can be diverted:

   require 'tempfile'

   tmp = Tempfile.new Process.pid
   puts tmp.path

   cmd = "put the exact command here"

   STDOUT.reopen tmp.path
   STDERR.reopen tmp.path

   exec cmd

this should dump everything into a tempfile. if it does, then you should be
able to manipulate the code with popen fine.

------
So yeah, the output from old-Link.exe is still being filtered up to
the top and seen. What I'm looking for is a way to shut it up... :wink:

alternatively, you may want to use systemu

   http://codeforpeople.com/lib/ruby/systemu

I looked at systemu and it's not clear how it would help with this
problem. :expressionless: Apologies if I'm being dense.

   harp:~ > cat a.rb
   require 'systemu'

   class Filter
     def initialize prefix, io
       @prefix = prefix
       @io = io
     end
     def << line
       @io << "#{ @prefix }: #{ line }"
     end
   end

   o = Filter.new 'stdout', STDOUT
   e = Filter.new 'stderr', STDERR

   systemu 'ruby -e"STDOUT.puts 42; STDERR.puts 42"', :stdout => o, :stderr => e

   harp:~ > ruby a.rb
   stdout: 42
   stderr: 42

-a

···

On Fri, 23 Feb 2007, Harold Hausman wrote:
--
be kind whenever possible... it is always possible.
- the dalai lama

5

>
> Hi Ara, thanks for taking the time to try and help me. What you're
> saying about stderr here sounds right to me, but I tried what you have
> above, and I'm still seeing the output I'm trying to 'squash' (read:
> block) ... Here is my code:
> #########################
> <snip code>
> #########################
>

hrrrm. this is a windozing type thing, but is it perhaps manipulating the
console directly? for instance, in unix, this can happen.

I believe that the underlying executable that I'm dealing with, namely
VC8's "link.exe" is doing something very dastardly.

it's hard to say without seeing your code, and the old code, but you could try
this to prove to yourself that stdout and stderr can be diverted:

   require 'tempfile'

   tmp = Tempfile.new Process.pid
   puts tmp.path

   cmd = "put the exact command here"

   STDOUT.reopen tmp.path
   STDERR.reopen tmp.path

   exec cmd

When I integrate the above code, a tempfile is indeed created, but it
remains empty. And what's more? I *still* see the output in the
console! (:

However, the following irb session indeed proves that stdout can be
redirected. (as well as makes me laugh out loud)

irb(main):001:0> require 'tempfile'
=> true
irb(main):002:0> tmp = Tempfile.new Process.pid
=> #<File:C:/DOCUME~1/Harold/LOCALS~1/Temp/484.484.0>
irb(main):003:0> STDOUT.reopen tmp.path

...
#hrhrhr.

   harp:~ > cat a.rb
   require 'systemu'

   class Filter
     def initialize prefix, io
       @prefix = prefix
       @io = io
     end
     def << line
       @io << "#{ @prefix }: #{ line }"
     end
   end

   o = Filter.new 'stdout', STDOUT
   e = Filter.new 'stderr', STDERR

   systemu 'ruby -e"STDOUT.puts 42; STDERR.puts 42"', :stdout => o, :stderr => e

   harp:~ > ruby a.rb
   stdout: 42
   stderr: 42

This code is also awesome, but when I integrate it into my tool, it
has literally no effect on the outcome.

All of this is leading me to believe that VC8's link.exe (or as I've
renamed it 'old-Link.exe') is doing something very tricky, not
involving stdout or stderr at all.

Moreover, I've exceeded the point where solving this problem will save
me time versus watching 200k of spewed linker warnings with each build
of my software. (of course due to a 3rd party library that wont give
me their .pdb files, and a bug in VC8 which causes /IGNORE: to be
*itself* ignored on the command line of the linker.)

Oh well, thanks again for the lessons, I know more about spawning
processes from Ruby on Windows than I ever thought I would.

Regards,
-Harold

···

On 2/23/07, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

On Fri, 23 Feb 2007, Harold Hausman wrote:

6

All of this is leading me to believe that VC8's link.exe (or as I've
renamed it 'old-Link.exe') is doing something very tricky, not
involving stdout or stderr at all.

Sorry if you've already covered this (I haven't read the whole thread.)

But, have you tried redirecting link.exe's output from a DOS prompt
yet?

  link blah > foo.out 2>&1

(Yes, surprisingly, cmd.exe supports 2>&1 syntax.)

Just wondered if even DOS is able to capture the output, or not?

Regards,

Bill

···

From: "Harold Hausman" <hhausman@gmail.com>

7

I believe that the underlying executable that I'm dealing with, namely
VC8's "link.exe" is doing something very dastardly.

it sure looks like it!

it's hard to say without seeing your code, and the old code, but you could
try this to prove to yourself that stdout and stderr can be diverted:

   require 'tempfile'

   tmp = Tempfile.new Process.pid
   puts tmp.path

   cmd = "put the exact command here"

   STDOUT.reopen tmp.path
   STDERR.reopen tmp.path

   exec cmd

When I integrate the above code, a tempfile is indeed created, but it
remains empty. And what's more? I *still* see the output in the
console! (:

However, the following irb session indeed proves that stdout can be
redirected. (as well as makes me laugh out loud)

irb(main):001:0> require 'tempfile'
=> true
irb(main):002:0> tmp = Tempfile.new Process.pid
=> #<File:C:/DOCUME~1/Harold/LOCALS~1/Temp/484.484.0>
irb(main):003:0> STDOUT.reopen tmp.path

hmm. then it looks like what's happening might depend on the nature of stdin,
i'd try one last thing

     require 'tempfile'

     i = Tempfile.new Process.pid
     o = Tempfile.new Process.pid
     e = Tempfile.new Process.pid
     puts i.path
     puts o.path
     puts e.path

     cmd = "put the exact command here"

     STDIN.reopen i.path
     STDOUT.reopen o.path
     STDERR.reopen e.path

     system cmd

alternatively

     require 'systemu'

     class Filter
       def initialize prefix, io
         @prefix = prefix
         @io = io
       end
       def << line
         @io << "#{ @prefix }: #{ line }"
       end
     end

     i = ''
     o = Filter.new 'stdout', STDOUT
     e = Filter.new 'stderr', STDERR

     systemu 'ruby -e"STDOUT.puts 42; STDERR.puts 42"', :stdin=>i, :stdout=>o, :stderr=>e

sorry i couldn't help more.

cheers.

-a

···

On Fri, 23 Feb 2007, Harold Hausman wrote:
--
be kind whenever possible... it is always possible.
- the dalai lama