i also saw that the ruby-dev suumary 23175-23213 by Masayoshi Takahashi spoke
of related issues…
…
[ruby-dev:23212] warning: fork terminates thread
Tanaka Akira proposed to suppress "fork terminates thread"
warnings on using folk with thread as follows:
% ruby -ve 'Thread.new { sleep }; pid = fork {}; Process.wait pid'
ruby 1.9.0 (2004-03-22) [i686-linux]
-e:1: warning: fork terminates thread at -e:1
Rationale for the proposal is:
* If the warning is for chang of specification of fork in
Ruby 1.8, it should be suppressed on Ruby 1.9.
* fork must be used when system and popen are inadequant.
In such a case, the warning is unsolicited.
* Even if the user doesn't use thread explicitly, some libraries
such like timeout use thread implicitly.
Matz agreed to do it.
…
my situation is that i have a long running peice of code that i need to start
in a thread (so the rest of a tk app can continue)… the code drives about 30
minutes of external processes for which i need the stdout and stderr. this is
made more difficult by the fact that i need to send multiple commands to the
external processes depending on their outputs (sort of expect like) and thus
cannot close the stdin of any of the processes… i’ve tried using select,
threads doing normal reads, etc - but any combination seems to result in the
thread getting all the output from the processes at once - not bit by bit.
this is obviously very undsirable for a gui which is supposed to be monitoring
the progress of these processes… starting the external processes in the
background via system and redirecting their stdin, stdout, stderr to temporary
fifos eliminates the fork and my problems… this little program, although
contrived, illustrates the problem:
if it uses open3 the output comes all at once, not incrementally
~/eg/ruby > spawn.rb open3
o @ 1080227752.76307: “42\n42\n”
if it uses spawn the output comes incrementally
~/eg/ruby > spawn.rb spawn
o @ 1080227758.40758: “42\n”
o @ 1080227760.00291: “42\n”
the code (sorry for length)
···
On Thu, 25 Mar 2004, Xavier wrote:
Nice idea IMHO.
Funny, I had to write something similar yesterday to get rid of open3
warnings that were triggered when publishing my objects with drb because
of its forks. This is what I did. It is much simpler as I do not need to
read stdout while the spawned process runs, nor write to its stdin. I only
need to get stderr and stdout separately.
def runrun(rcmd)
Get a temp file
tmp = Tempfile.new(“runrun”)
Redirect stderr into temp file
po = IO.popen(“#{rcmd} 2>#{tmp.path}”)
stdo = po.readlines
po.close
Reopen temp file and read STDERR
tmp.open
stdr = tmp.readlines
Close and unlink temp file
tmp.close!
return STDOUT and STDERR
[stdo, stdr]
end
Regards,
Xavier.
#!/usr/bin/env ruby
require ‘open3’
require ‘tmpdir’
require ‘io/wait’
$VERBOSE=nil
module Spawn
class << self
def spawn cmd
ipath, opath, epath = tmpfifo, tmpfifo, tmpfifo
system “#{ cmd } < #{ ipath } 1> #{ opath } 2> #{ epath } &”
[open(ipath, ‘w’), open(opath, ‘r’), open(epath, ‘r’)]
end
def tmpfifo
path = nil
42.times do |i|
tpath = File.join(Dir.tmpdir, “#{ $$ }.#{ rand }.#{ i }”)
system “mkfifo #{ tpath }”
next unless $? == 0
path = tpath
at_exit{ File.unlink(path) rescue STDERR.puts(“rm <#{ path }> failed”) }
break
end
raise “could not generate tmpfifo” unless path
path
end
end
end
mode = ARGV.shift || ‘open3’
program = <<-stmts
echo 42
sleep 1
echo 42
sleep 1
echo done # signal end of stdout
echo done 1>&2 # signal end of stderr
stmts
done = %r/^\s*done\s*$/o
thread =
Thread.new do
i,o,e = (mode =~ /open3/io ? Open3::popen3(‘sh’) : Spawn::spawn(‘sh’))
program.each{|stmt| i.puts stmt}
i.flush
rfds = [o,e]
loop do
break if rfds.empty?
rs, = select rfds, nil, nil
rs.each do |r|
rfds.delete(r) and next if r.eof?
line = ''
line << r.getc while r.ready?
rfds.delete(r) and line[done]='' if line =~ done
next if line.empty?
printf "%s @ %s: %s\n", (r == o ? 'o' : 'e'), Time.now.to_f, line.inspect
end
end
end
thread.join
-a
===============================================================================
EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================
i’m not suprised…