Processes and Portability

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

This thus eliminates the use of fork. Also, I have tried:

a=IO.popen("program > logfile"), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

So I require portability in that sense.

Is this possible?

thanks

···

--
Posted via http://www.ruby-forum.com/.

try this - i haven't tested on windows but it can certainly be made to work:

harp:~ > cat a.rb
#!/usr/bin/env ruby

require 'rbconfig'
require 'thread'

class Spawn
   CONFIG = Config::CONFIG

   RUBY =
     if system('ruby -e 42') # ruby is in our path
       'ruby'
     else # guess from rbconfig
       File.join CONFIG['bindir'], CONFIG['ruby_install_name']
     end

   PROGRAM = "p $$;STDOUT.flush;STDERR.reopen(STDOUT);exec(*ARGV)"

   ATTRIBUTES = %w[
     mutex
     out
     exitstatus
     exitstatus_q
     cmd
     pipe
     pid
     thread
   ]

   ATTRIBUTES.each{|a| attr_accessor a}

   def initialize *argv
     @mutex = Mutex.new
     @out = ''
     @exitstatus = nil
     @exitstatus_q = Queue.new
     @cmd = "#{ RUBY } -e '#{ PROGRAM }' #{ argv.join(' ') }"
     @pipe = IO.popen @cmd
     @pid = @pipe.gets.to_i
     at_exit{ Process.kill -9, @pid rescue nil }
     @thread = Thread.new{
       begin
         while((line = @pipe.gets))
           @mutex.synchronize{ @out << line }
         end
       ensure
         @pipe.close rescue nil
         exitstatus = $?.exitstatus
         @exitstatus_q.push exitstatus
       end
     }
   end

   def out
     @mutex.synchronize{ @out }
   end

   def exitstatus non_block = false
     @exitstatus ||= @exitstatus_q.pop(non_block)
   end

   class << self; alias new; end
end

require 'yaml'

cmd = ARGV.join ' '

spawn = Spawn[ cmd ]

y 'spawn.cmd' => spawn.cmd
y 'spawn.pid' => spawn.pid
y 'spawn.exitstatus' => spawn.exitstatus
y 'spawn.out' => spawn.out

harp:~ > ruby a.rb date
spawn.cmd: ruby -e 'p $$;STDOUT.flush;STDERR.reopen(STDOUT);exec(*ARGV)' date
spawn.pid: 27232
spawn.exitstatus: 0
spawn.out: |
   Tue Jul 11 10:31:22 MDT 2006

harp:~ > ruby a.rb no-exist
spawn.cmd: ruby -e 'p $$;STDOUT.flush;STDERR.reopen(STDOUT);exec(*ARGV)' no-exist
spawn.pid: 27235
spawn.exitstatus: 1
spawn.out: |
   -e:1:in `exec': No such file or directory - no-exist (Errno::ENOENT)
         from -e:1

regards.

-a

···

On Wed, 12 Jul 2006, Name Name wrote:

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process) and
have all standard out of that process be put into a text file. I would also
like this code to be portable. And finally, I would like to be able to kill
this process at any time (So I will need the pid)

This thus eliminates the use of fork. Also, I have tried:

a=IO.popen("program > logfile"), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

So I require portability in that sense.

Is this possible?

thanks

--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

This thus eliminates the use of fork. Also, I have tried:

If you are doing the equivalent of fork/exec, here's some code that is in my IOWA package that will help. It depends on Daniel Berger's win32-process library, which can be found at RubyForge:

http://rubyforge.org/projects/win32utils

It, in turn, is dependent on window-pr, found at the same URL.

This code has been tested on Linux and WinXP. You should be able to adapt it to your purposes.

module IWATestSupport
   def self.create_process(args)
     @fork_ok = true unless @fork_ok == false
     pid = nil
     begin
       raise NotImplementedError unless @fork_ok
       unless pid = fork
         Dir.chdir args[:dir]
         exec(*args[:cmd])
       end
     rescue NotImplementedError
       @fork_ok = false

       begin
         require 'rubygems'
       rescue LoadError
       end

       begin
         require 'win32/process'
       rescue LoadError
         raise "Please install win32-process."
       end

       cwd = Dir.pwd
       Dir.chdir args[:dir]
       pid = Process.create(:app_name => args[:cmd].join(' '))
       Dir.chdir cwd
     end
     pid
   end
end

Kirk Haines

Name Name <exl2@cornell.edu> writes:

a=IO.popen("program > logfile"), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

The behaviour in *nix depends on your /bin/sh. Some shells gives the
child's pid, other gives the pid of itself.

pdksh gives the pid of itself.
bash gives the pid of the child (the behaviour you wanted).
Don't know about other shells.

The lesson here, you can't get the pid of the child consistently
unless you do the forking yourself.

YS

Name Name wrote:

I would like to run a process concurrently (thus if the process goes to sleep or runs forever, I can still run other code from my main process) and have all standard out of that process be put into a text file. I would also like this code to be portable. And finally, I would like to be able to kill this process at any time (So I will need the pid)

I was half expecting the following to work, but it doesn't:

   t = Thread.new do
     system "sleep 100000" # or IO.popen or fork...
   end

   puts "press return to stop child thread"
   gets

   t.kill

   puts `ps -Af | grep sleep`
   Process.wait # still waiting for child until ^C

Killing a thread that is waiting on system() doesn't stop the child process.

Should this be part of the Thread abstraction? Should threads own processes? Maybe not. Just wondering what people think...

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

I know this sounds nieve, but is there a way to do what I want without
requiring an external gem, and without that extremely long code on the
secnod post. I would really prefer just to use native ruby.

Be able to concurrently run a process (that could potentially block
forever or run forever) written in another language and having the
ability to stop it at anytime.

a=IO.popen (program_name)
Process.kill(9,a.pid)

works, but it won't log to a file

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

···

--
Posted via http://www.ruby-forum.com/.

I want this to be portable, ie not just *nix, so I can't fork. However,
I am doing something similar to forking in terms of getting the pid
which popen will do. I'm sure if I used fork (which I can't) on program

logfile, it won't be the pid of program (or it might be, i'm not sure,

but the point is, the logic is messed up since I am doign the extra
redirecting)

Yohanes Santoso wrote:

Name Name <exl2@cornell.edu> writes:

a=IO.popen("program > logfile"), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

The behaviour in *nix depends on your /bin/sh. Some shells gives the
child's pid, other gives the pid of itself.

pdksh gives the pid of itself.
bash gives the pid of the child (the behaviour you wanted).
Don't know about other shells.

The lesson here, you can't get the pid of the child consistently
unless you do the forking yourself.

YS

I know that Harmen. I want the pid of not the intermediate shell, which
is the one I am getting back with popen. I want the pid of the program
itself that I opened. Also, program does produce output and logfile is
writeable, and I need this to run on any system.

···

On Linux that should work if program 'progam_name' produces output and
file 'logfile' is writable. One of the two processes you mentioned is an
intermediate shell by the way.

Harmen

--
Posted via http://www.ruby-forum.com/\.

Joel, if you do t.status after you thread off the system, you'll notice
that it says sleeping instead of running. Pretty much the thread will
make the system call and then wait on that system call to return. So
that thread is owning that process in some respects, but killing that
thread will in no means kill the process, since that process is on its
own now.

···

On an unrelated note, does anyone know how to kill all children pids including itself when one is given a pid? If that question is answered, I think my post will be answered. Joel VanderWerf wrote:

Name Name wrote:

I would like to run a process concurrently (thus if the process goes to
sleep or runs forever, I can still run other code from my main process)
and have all standard out of that process be put into a text file. I
would also like this code to be portable. And finally, I would like to
be able to kill this process at any time (So I will need the pid)

I was half expecting the following to work, but it doesn't:

   t = Thread.new do
     system "sleep 100000" # or IO.popen or fork...
   end

   puts "press return to stop child thread"
   gets

   t.kill

   puts `ps -Af | grep sleep`
   Process.wait # still waiting for child until ^C

Killing a thread that is waiting on system() doesn't stop the child
process.

Should this be part of the Thread abstraction? Should threads own
processes? Maybe not. Just wondering what people think...

--
Posted via http://www.ruby-forum.com/\.

or is there is a way to directly write to a file using IO.popen.

···

--
Posted via http://www.ruby-forum.com/.

Name Name wrote:

...

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

   a=IO.popen("#{program_name} > #{logfile}")
   Process.kill(9, a.pid)

Don't know if that works on MSWindows.

Harmen

···

--
Posted via http://www.ruby-forum.com/\.

so i was just testing right now using process explorer that:

Process.kill(9, a.pid) only kills the parent and detaches the child,
letting the child persist.

I also noticed that if i right click on the parent and click kill tree,
it will kill both on process explorer.

how do i kill the parent and the children, not jsut that parent as in
the case above.

···

--
Posted via http://www.ruby-forum.com/.

Name Name <exl2@cornell.edu> writes:

I want this to be portable, ie not just *nix, so I can't fork.

It is because of portability issue that I wrote my reply.
Being portable does not mean staying of OS-specific features. You can
do that, if there is another option for you to take. In this case,
however, there is no other.

So, being portable for your needs means providing different workaround
for different environment.

I don't know much about win32, but I know for *nix, you need to do the
forking and setup IO redirection yourself (not using IO.popen at all).

The other people here have replied about doing this in win32.

YS.

···

However, I am doing something similar to forking in terms of getting the pid
which popen will do. I'm sure if I used fork (which I can't) on program

logfile, it won't be the pid of program (or it might be, i'm not sure,

but the point is, the logic is messed up since I am doign the extra
redirecting)

Yohanes Santoso wrote:

Name Name <exl2@cornell.edu> writes:

a=IO.popen("program > logfile"), but on Windows this creates two
processes, cmd.exe and program. a.pid only gives the pid of cmd.exe.
And the behavior on *nix seems to be just the program (although I can be
wrong).

The behaviour in *nix depends on your /bin/sh. Some shells gives the
child's pid, other gives the pid of itself.

pdksh gives the pid of itself.
bash gives the pid of the child (the behaviour you wanted).
Don't know about other shells.

The lesson here, you can't get the pid of the child consistently
unless you do the forking yourself.

YS

I know that Harmen. I want the pid of not the intermediate shell, which
is the one I am getting back with popen. I want the pid of the program
itself that I opened. Also, program does produce output and logfile is
writeable, and I need this to run on any system.

On Linux that should work if program 'progam_name' produces output and
file 'logfile' is writable. One of the two processes you mentioned is an
intermediate shell by the way.

Harmen

--
Posted via http://www.ruby-forum.com/\.

harp:~ > cat a.rb
   Thread.new{ system 'sleep 60' }
   sleep 1
   system 'ps'
   Process.kill -9, Process.pid

   harp:~ > ruby a.rb
     PID TTY TIME CMD
   15149 pts/2 00:00:00 bash
    6308 pts/2 00:00:00 ruby
    6309 pts/2 00:00:00 sleep
    6310 pts/2 00:00:00 ps
   Killed

   harp:~ > ps
     PID TTY TIME CMD
   15149 pts/2 00:00:00 bash
    6311 pts/2 00:00:00 ps

-a

···

On Wed, 12 Jul 2006, Name Name wrote:

Joel, if you do t.status after you thread off the system, you'll notice
that it says sleeping instead of running. Pretty much the thread will
make the system call and then wait on that system call to return. So
that thread is owning that process in some respects, but killing that
thread will in no means kill the process, since that process is on its
own now.

On an unrelated note, does anyone know how to kill all children pids
including itself when one is given a pid? If that question is answered,
I think my post will be answered.

--
suffering increases your inner strength. also, the wishing for suffering
makes the suffering disappear.
- h.h. the 14th dali lama

Name Name wrote:

Joel, if you do t.status after you thread off the system, you'll notice that it says sleeping instead of running. Pretty much the thread will make the system call and then wait on that system call to return. So that thread is owning that process in some respects, but killing that thread will in no means kill the process, since that process is on its own now.

Sure, that's why the Process.wait call hangs the program until you interrupt it.

The thread owns the process in one sense: the thread continues (and in this case finishes) when the process exits. I'm just wondering whether it would make sense for the thread to be more closely associated with the process. Maybe killing the thread should not kill the process, but instead there should be an additional status result that tells you whether the thread is sleeping waiting for an external process and if so the command line and the pid.

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

I don't understand how that is different than:
a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

Harmen Schut wrote:

···

Name Name wrote:

...

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

   a=IO.popen("#{program_name} > #{logfile}")
   Process.kill(9, a.pid)

Don't know if that works on MSWindows.

Harmen

--
Posted via http://www.ruby-forum.com/\.

Is there a command in ruby that when given the pid , it will terminate
all children processes of that pid including itself.

thanks

Yohanes Santoso wrote:

···

Name Name <exl2@cornell.edu> writes:

I want this to be portable, ie not just *nix, so I can't fork.

It is because of portability issue that I wrote my reply.
Being portable does not mean staying of OS-specific features. You can
do that, if there is another option for you to take. In this case,
however, there is no other.

So, being portable for your needs means providing different workaround
for different environment.

I don't know much about win32, but I know for *nix, you need to do the
forking and setup IO redirection yourself (not using IO.popen at all).

The other people here have replied about doing this in win32.

YS.

--
Posted via http://www.ruby-forum.com/\.

It is one String argument (a command line) instead of an expression
(compairing two String variables?)

Harmen

Name Name wrote:

···

I don't understand how that is different than:
a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

Harmen Schut wrote:

Name Name wrote:

...

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

   a=IO.popen("#{program_name} > #{logfile}")
   Process.kill(9, a.pid)

Don't know if that works on MSWindows.

Harmen

--
Posted via http://www.ruby-forum.com/\.

I'm sorry if my previous thing was confusing, but it was actually:

IO.popen("program_name > logfile").

Harmen Schut wrote:

···

It is one String argument (a command line) instead of an expression
(compairing two String variables?)

Harmen

Name Name wrote:

I don't understand how that is different than:
a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

Harmen Schut wrote:

Name Name wrote:

...

a=IO.popen(program_name > logfile)
Process.kill(9, a.pid)

spawns two processes and does not kill both.

I need something to be able to do the latter.

thanks

What about:

   a=IO.popen("#{program_name} > #{logfile}")
   Process.kill(9, a.pid)

Don't know if that works on MSWindows.

Harmen

--
Posted via http://www.ruby-forum.com/\.

On Linux that should work if program 'progam_name' produces output and
file 'logfile' is writable. One of the two processes you mentioned is an
intermediate shell by the way.

Harmen

Name Name wrote:

···

I'm sorry if my previous thing was confusing, but it was actually:

IO.popen("program_name > logfile").

...

--
Posted via http://www.ruby-forum.com/\.