Interactively processing a line at a time

ruby-talk
1

HI All:

        For the following code segement running on win32 ruby (1 click installer)

sout = IO.popen( "output");
while( line = sout.gets)
  do_something_with( line)
end

        If for example, "output" printed out something 5 times every second,
then on the command line, I would see something printed out every
second, 5 times. Running through the above code, I would not see
anything until after 5 secs, which I assume that after the program
exits, the output is flushed, and then it would be all 5 entries. I
ran the same sort of code successfully solaris, so I expected it to
work on windows.

        I did an experiment, and wrote a ruby program that printed a
line then flushed STDOUT, paused, repeated, etc., and it works as
suspected. Since I do not have control over the actual "output"
program, how can I get it so that each line is flushed? sout.sync =
true didn't work (I imagine it only works if I write to sout)

        Any ideas?

        Thanks

        FOrrest

2

try this - it should/might work on windoze - i assume it inherits open file
handles from the parent. i ran it on xp and it seemed fine:

     harp:~ > ruby a.rb
     [Fri Aug 05 20:49:51 MDT 2005, "42\n"]
     [Fri Aug 05 20:49:56 MDT 2005, "42\n"]
     [Fri Aug 05 20:50:01 MDT 2005, "42\n"]
     [Fri Aug 05 20:50:06 MDT 2005, "42\n"]
     [Fri Aug 05 20:50:11 MDT 2005, "42\n"]
     ...

     harp:~ > cat a.rb
     require 'tempfile'

     def popen_sync cmd
       tf = Tempfile::new $PROGRAM_NAME
       tf.puts <<-ruby_code
         STDOUT.sync = true
         exec STDIN.read
       ruby_code
       tf.close

       sync_cmd = "ruby #{ tf.path }"
       pipe = IO::popen sync_cmd, 'r+'
       pipe.puts cmd
       pipe.close_write
       pipe
     end

     output = "ruby -e 'loop{ p 42; sleep 5}'"
     sout = popen_sync output

     while(line = sout.gets)
      p [Time::now, line]
      STDOUT.flush
     end

again - this will only work if exce inherits open files and there properties...

hth.

-a

···

On Sat, 6 Aug 2005, Forrest Chang wrote:

HI All:

       For the following code segement running on win32 ruby (1 click installer)

sout = IO.popen( "output");
while( line = sout.gets)
do_something_with( line)
end

       If for example, "output" printed out something 5 times every second,
then on the command line, I would see something printed out every
second, 5 times. Running through the above code, I would not see
anything until after 5 secs, which I assume that after the program
exits, the output is flushed, and then it would be all 5 entries. I
ran the same sort of code successfully solaris, so I expected it to
work on windows.

       I did an experiment, and wrote a ruby program that printed a
line then flushed STDOUT, paused, repeated, etc., and it works as
suspected. Since I do not have control over the actual "output"
program, how can I get it so that each line is flushed? sout.sync =
true didn't work (I imagine it only works if I write to sout)

       Any ideas?

--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
My religion is very simple. My religion is kindness.
--Tenzin Gyatso

===============================================================================

3

"Ara.T.Howard" <Ara.T.Howard@noaa.gov> writes:

HI Ara:

   That did the trick. Thanks.

   BTW, nice article on Grids w/RUby and SQLite

Forrest

···

On Sat, 6 Aug 2005, Forrest Chang wrote:

HI All:

       For the following code segement running on win32 ruby (1 click installer)

sout = IO.popen( "output");
while( line = sout.gets)
do_something_with( line)
end

       If for example, "output" printed out something 5 times every second,
then on the command line, I would see something printed out every
second, 5 times. Running through the above code, I would not see
anything until after 5 secs, which I assume that after the program
exits, the output is flushed, and then it would be all 5 entries. I
ran the same sort of code successfully solaris, so I expected it to
work on windows.

       I did an experiment, and wrote a ruby program that printed a
line then flushed STDOUT, paused, repeated, etc., and it works as
suspected. Since I do not have control over the actual "output"
program, how can I get it so that each line is flushed? sout.sync =
true didn't work (I imagine it only works if I write to sout)

       Any ideas?

try this - it should/might work on windoze - i assume it inherits open file
handles from the parent. i ran it on xp and it seemed fine:

     harp:~ > ruby a.rb
     [Fri Aug 05 20:49:51 MDT 2005, "42\n"]
     [Fri Aug 05 20:49:56 MDT 2005, "42\n"]
     [Fri Aug 05 20:50:01 MDT 2005, "42\n"]
     [Fri Aug 05 20:50:06 MDT 2005, "42\n"]
     [Fri Aug 05 20:50:11 MDT 2005, "42\n"]
     ...
     ...
     ...

     harp:~ > cat a.rb
     require 'tempfile'

     def popen_sync cmd
       tf = Tempfile::new $PROGRAM_NAME
       tf.puts <<-ruby_code
         STDOUT.sync = true
         exec STDIN.read
       ruby_code
       tf.close

       sync_cmd = "ruby #{ tf.path }"
       pipe = IO::popen sync_cmd, 'r+'
       pipe.puts cmd
       pipe.close_write
       pipe
     end

     output = "ruby -e 'loop{ p 42; sleep 5}'"
     sout = popen_sync output

     while(line = sout.gets)
      p [Time::now, line]
      STDOUT.flush
     end

again - this will only work if exce inherits open files and there properties...

hth.

-a
--

> email :: ara [dot] t [dot] howard [at] noaa [dot] gov
> phone :: 303.497.6469
> My religion is very simple. My religion is kindness.
> --Tenzin Gyatso

4

"Ara.T.Howard" <Ara.T.Howard@noaa.gov> writes:

Hi Ara:

   I tried tweaking your code to use IO.win32_popen3 coz I wanted to
process stderr as well and we're back to the "doesn't look like
stdout/stderr are being flushed". Any additional ideas?

   Thanks again

   Forrest

5

"Ara.T.Howard" <Ara.T.Howard@noaa.gov> writes:

HI Ara:

  That did the trick. Thanks.

great. i wouldn't launch rockets with it :wink:

  BTW, nice article on Grids w/RUby and SQLite

thanks! it's been really good for us - millions of jobs run w/o a single
incident or bug for nearly 9 months of 24x7 operation - it's amazing what a
garbage collector can do :wink:

cheers.

-a

···

On Sat, 13 Aug 2005, Forrest Chang wrote:
--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
Your life dwells amoung the causes of death Like a lamp standing in a strong breeze. --Nagarjuna

===============================================================================

6

here's something using open4:

   require 'tempfile'
   require 'open4'

   def maim pid, opts = {}
#--{{{
     sigs = opts['signals'] || opts[:signals] || %w(SIGTERM SIGQUIT SIGKILL)
     suspend = opts['suspend'] || opts[:suspend] || 4
     pid = Integer "#{ pid }"
     existed = false
     sigs.each do |sig|
       begin
         Process::kill sig, pid
         existed = true
       rescue Errno::ESRCH
         unless existed
           return nil
         else
           return true
         end
       end
       return true unless alive?(pid)
       sleep suspend
       return true unless alive?(pid)
     end
     return(not alive?(pid)) #--}}}
   end
   def alive? pid
#--{{{
     pid = Integer "#{ pid }"
     begin
       Process::kill 0, pid
       true
     rescue Errno::ESRCH
       false
     end
#--}}}
   end
   def popen_sync cmd
#--{{{
     tf = Tempfile::new $PROGRAM_NAME
     tf.puts <<-ruby_code
       STDOUT.sync = STDERR.sync = true
       exec STDIN.read
     ruby_code
     tf.close

     sync_cmd = "ruby #{ tf.path }"

     pid, stdin, stdout, stderr = Open4::popen4 sync_cmd

     stdin.puts cmd
     stdin.close

     if block_given?
       begin
         yield [pid, stdout, stderr]
       ensure
         maim pid
       end
     else
       at_exit{ maim pid rescue nil }
       [pid, stdout, stderr]
     end
#--}}}
   end

   output_program = "ruby -e 'loop{ p 42; sleep 5}'"

   popen_sync(output_program) do |pid, stdout, stderr|
     while(line = stdout.gets)
      puts "stdout => <#{ line.strip }>"
      STDOUT.flush
     end
     while(line = stderr.gets)
      puts "stderr => <#{ line.strip }>"
      STDOUT.flush
     end
   end

handling stdout and stderr simoultaneously on windows will be tricky though...
see many threads on this list for that.

cheers.

-a

···

On Sat, 13 Aug 2005, Forrest Chang wrote:

"Ara.T.Howard" <Ara.T.Howard@noaa.gov> writes:

Hi Ara:

  I tried tweaking your code to use IO.win32_popen3 coz I wanted to
process stderr as well and we're back to the "doesn't look like
stdout/stderr are being flushed". Any additional ideas?

  Thanks again

  Forrest

--

email :: ara [dot] t [dot] howard [at] noaa [dot] gov
phone :: 303.497.6469
Your life dwells amoung the causes of death Like a lamp standing in a strong breeze. --Nagarjuna

===============================================================================