Timeout lengthy external executable

ruby-talk
1

Hi all,
I'm new to Ruby, I'm trying to use it to create a script where I launch
some executables.
I'd like to timeout if one of them takes too long, I'm using IO.popen to
launch the exe.
The problem is that the timeout doesn't seem to work, so if my
executable hangs for some reason I have no way to stop it and go on.

Is there something I can do?
If I use "system" instead of IO.popen it works but I need popen to parse
stdout

I include a simplified code of what I'm doing (no parsing)

Thanks in advance for any tips on that
Bye

begin
    timeout(5) {
      IO.popen(command_media)
      }
     puts "Timeout didn't occur"
rescue Timeout::Error
     puts "Timed out!"
end

···

--
Posted via http://www.ruby-forum.com/.

2

if you're on windows try using 'systemu', which will allow you to capture stdout/stderr and should also should not block your app.

gem install systemu

http://codeforpeople.com/lib/ruby/systemu/systemu-1.2.0/README

a @ http://codeforpeople.com/

···

On Jun 15, 2008, at 10:32 AM, Ema Fuma wrote:

Hi all,
I'm new to Ruby, I'm trying to use it to create a script where I launch
some executables.
I'd like to timeout if one of them takes too long, I'm using IO.popen to
launch the exe.
The problem is that the timeout doesn't seem to work, so if my
executable hangs for some reason I have no way to stop it and go on.

Is there something I can do?
If I use "system" instead of IO.popen it works but I need popen to parse
stdout

I include a simplified code of what I'm doing (no parsing)

Thanks in advance for any tips on that
Bye

begin
   timeout(5) {
     IO.popen(command_media)
     }
    puts "Timeout didn't occur"
rescue Timeout::Error
    puts "Timed out!"
end

--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

3

gem install systemu

http://codeforpeople.com/lib/ruby/systemu/systemu-1.2.0/README

a @ http://codeforpeople.com/

Hi thanks for the quick answer,

I'm new to ruby and RubyGems, it seemed that the installation was
succesful
but I get an error when requiring the file. I'll try to figure it how,
what I would like to ask is that if with systemu I will be able to parse
the stdout and stderr line by line at real time (not waiting the process
to finish to parse stdout). From the doc it's not too clear to me

Thanks again

error when launching:
encode7.rb:92:in `require': no such file to load -- systemu (LoadError)
  from encode7.rb:92

···

--
Posted via http://www.ruby-forum.com/\.

4

I'm new to ruby and RubyGems, it seemed that the installation was
succesful
but I get an error when requiring the file. I'll try to figure it how,

require 'rubygems'
require 'systemu'

what I would like to ask is that if with systemu I will be able to parse
the stdout and stderr line by line at real time (not waiting the process
to finish to parse stdout). From the doc it's not too clear to me

you cannot do this on windows safely. if you are on *nix you can easily parse in realtime using the open4 lib, which is much more powerful than systemu. however, only systemu is cross platform.

cheers.

a @ http://codeforpeople.com/

···

On Jun 15, 2008, at 11:22 AM, Ema Fuma wrote:
--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

5

require 'rubygems'
require 'systemu'

Thanks, with that it worked perfectly!

you cannot do this on windows safely. if you are on *nix you can
easily parse in realtime using the open4 lib, which is much more
powerful than systemu. however, only systemu is cross platform.

Yes, I tried systemu and work exactly as you explained.
Yes, for me it would be better to have my script cross-platform
(providing for each OS the compiled executables to launch).
With IO.popen I was able to parse at run-time line by line the stdout
like this:

           IO.popen(command) do |pipe|
        pipe.each("\n") do |line|
        puts line
      end

And it's working (Windows and OSX), the only problem I had was that
timeout(10) wasn't ignored.
Now what I want to achieve by the run-time parsing are two things:
- display a progess of what is happening
- check that the process keeps writing on stdout, so I know it's still
alive. If it doesn't for 10 sec. exit the entire program.

Is that possible?

Thanks a lot again for your kindness!

···

--
Posted via http://www.ruby-forum.com/\.

6

i'm 99% positive that trying to read from the pipe will conflict with the timeout method - because of the way ruby uses green threads it's easy to lock up threads where io is concerned. sometimes select can be used but it's a case by case basis for mixing threads, io, and non-blocking behaviour on windows. YMMV.

cheers.

a @ http://codeforpeople.com/

···

On Jun 15, 2008, at 12:00 PM, Ema Fuma wrote:

Yes, I tried systemu and work exactly as you explained.
Yes, for me it would be better to have my script cross-platform
(providing for each OS the compiled executables to launch).
With IO.popen I was able to parse at run-time line by line the stdout
like this:

          IO.popen(command) do |pipe|
       pipe.each("\n") do |line|
       puts line
     end

And it's working (Windows and OSX), the only problem I had was that
timeout(10) wasn't ignored.
Now what I want to achieve by the run-time parsing are two things:
- display a progess of what is happening
- check that the process keeps writing on stdout, so I know it's still
alive. If it doesn't for 10 sec. exit the entire program.

Is that possible?

--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

7

i'm 99% positive that trying to read from the pipe will conflict with
the timeout method - because of the way ruby uses green threads it's
easy to lock up threads where io is concerned. sometimes select can
be used but it's a case by case basis for mixing threads, io, and non-
blocking behaviour on windows. YMMV.

Ok, I thought I was doing something wrong, or not using the right
method.
Maybe I should use some customized timeout?
To me it looked strange that in a scripting language like Ruby it was
impossible to timeout certain operations and I thought someone else
found a solution.
Thanks

bye

···

--
Posted via http://www.ruby-forum.com/\.

8

it's not ruby per-se, but the interaction of select on windows. you haven't yet said - what platform are you on?

a @ http://codeforpeople.com/

···

On Jun 16, 2008, at 1:25 AM, Ema Fuma wrote:

Ok, I thought I was doing something wrong, or not using the right
method.
Maybe I should use some customized timeout?
To me it looked strange that in a scripting language like Ruby it was
impossible to timeout certain operations and I thought someone else
found a solution.
Thanks

bye

--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

9

it's not ruby per-se, but the interaction of select on windows. you
haven't yet said - what platform are you on?

I'm under Windows but I would like to use the same script on Linux.
Basically it's just
-launch an external executable that write it's progress on stdout
-parse the stdout line by line
-if there's no new line for more than 10 sec close the external exe

Thanks

···

--
Posted via http://www.ruby-forum.com/\.

10

it's nearly impossible to do this on windows reliably. this are just too many failure conditions, for instance say the child program does

   gb = 2 ** 30

   huge = '*' * gb

   STDOUT.write huge

and the parent does

   buf = child.gets

both processes will hang. the child because it fills the pipe, the parent because it never sees a newline. you'd think this could be solved by using timeout, but it cannot (on windows) because ruby's threads schedule with select and therefore many/most io operations will block the scheduler - causing a hang.

there is some hope for using socketpair and a tcp server in a cross platoform manner, rough idea here:

   http://pastie.org/173246

but this doesn't address issues like stderr or zombie processes.

anyhow, i'm not saying it *can't* be done on windows, but it's very non-trival to combine pipes, timeout, and process management in a robust fashion.

one idea might be to launch and external process to signal yourself before reading, something like this:

   signaler = Thread.new do
     system "ruby -e' sleep 10; Process.kill(:HUP.to_s, #{ Process.pid })"
   end

   begin
     child.gets
   rescue SignalException
     ...
   end

however signals are very limited on windows and i've no idea if even the signal handlers would fire once a process is blocked - worth trying though... it's definitely going to take something creative on your end. one other thought is to spawn and external reaper before reading, one which would kill the external exe for you

   reaper = Thread.new
     system "ruby -e' sleep 10; Process.kill -9, #{ child.pid }'"
   end

and then, in the parent arrange to schedule this before each read, and kill it afterwards...

i'd be interested to hear other's thoughts on the issue (dan berger in the house?) as i'm no windows expert.

regards.

a @ http://codeforpeople.com/

···

On Jun 16, 2008, at 10:41 AM, Ema Fuma wrote:

I'm under Windows but I would like to use the same script on Linux.
Basically it's just
-launch an external executable that write it's progress on stdout
-parse the stdout line by line
-if there's no new line for more than 10 sec close the external exe

--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

11

Thanks for the very detailed explanation.
This is my first project using Ruby, I didn't expect that doing this
quite simple taks was so difficult, if I have to resort to signals and
threads maybe it would be better for me to do that in C++ where I'm more
experienced.
Thanks again

···

--
Posted via http://www.ruby-forum.com/.