I tried to write a script that makes use of external binaries. Each
external binary is called from a different thread, but, under 1.8.6,
this doesn't seem to work. Everything is executing in a sequential-like
manner.
The guys from StackOverflow said this was because of Ruby's thread
implementation ( and when I tried the same code under JRuby, it worked
).
Is there some way to make my script work under 1.8.6, or is upgrading to
1.9 the only solution?
I tried to write a script that makes use of external binaries. Each
external binary is called from a different thread, but, under 1.8.6,
this doesn't seem to work. Everything is executing in a sequential-like
manner.
The guys from StackOverflow said this was because of Ruby's thread
implementation ( and when I tried the same code under JRuby, it worked
).
Is there some way to make my script work under 1.8.6, or is upgrading to
1.9 the only solution?
What do you mean by external binary? Something you invoke with #system or the like?
I don't think that is necessary. Can you provide a example of the
phenomenon you describe? Normally, external programs are separate
processes and work independently. It may be though that if you do the
IO handling for external processes not properly that it looks like
they are executed sequentially because they are blocked in IO
operations.
Kind regards
robert
···
2009/9/7 vhaerun vh <etaern@yahoo.com>:
I tried to write a script that makes use of external binaries. Each
external binary is called from a different thread, but, under 1.8.6,
this doesn't seem to work. Everything is executing in a sequential-like
manner.
The guys from StackOverflow said this was because of Ruby's thread
implementation ( and when I tried the same code under JRuby, it worked
).
Is there some way to make my script work under 1.8.6, or is upgrading to
1.9 the only solution?
Stricly spoken it's not blocking. The "baz" is written to its pipe
before "bar". But it is stored until "quux" is echoed and puts is
asked to write it out.
Bertram
···
Am Montag, 07. Sep 2009, 20:06:32 +0900 schrieb Eleanor McHugh:
Stricly spoken it's not blocking. The "baz" is written to its pipe
before "bar". But it is stored until "quux" is echoed and puts is
asked to write it out.
The commands executed in the script at the SO link was simply pinging
the host on an ip range from 1 to 254 like in the following line :
`ping #{ip_addr}`
This happened from inside each individual thread. How could I replace
the backticks so that they won't block ?
You can use one of the popen methods, e.g.
IO.popen ["ping", ip_addr] do |io|
io.each {|l| puts l}
end
Example
15:37:19 ~$ ruby <<EOF
threads = (1..2).map do |i|
Thread.new i do |ii|
IO.popen 'ping 192.168.110.74' do |io|
io.each do |line|
printf "%2d %s", ii, line
end
end
end
end
threads.map do |th|
th.value
end
EOF
1
1 Pinging 192.168.110.74 with 32 bytes of data:
1
1 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
2
2 Pinging 192.168.110.74 with 32 bytes of data:
2
2 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
1 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
2 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
1 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
2 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
1 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
1
1 Ping statistics for 192.168.110.74:
1 Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
1 Approximate round trip times in milli-seconds:
1 Minimum = 0ms, Maximum = 0ms, Average = 0ms
2 Reply from 192.168.110.74: bytes=32 time<1ms TTL=128
2
2 Ping statistics for 192.168.110.74:
2 Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
2 Approximate round trip times in milli-seconds:
2 Minimum = 0ms, Maximum = 0ms, Average = 0ms
15:37:38 ~$ ruby -version
ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]
-e:1: undefined local variable or method `rsion' for main:Object (NameError)
15:37:53 ~$
Stricly spoken it's not blocking. The "baz" is written to its pipe
before "bar". But it is stored until "quux" is echoed and puts is
asked to write it out.
From Ruby's perspective the backtick is definitely a blocking IO operation,
just as it is in shell script.
Yes, of course. I just examined the interpreter's source. The
waitpid function is called _without_ the WNOHANG flag. Before that
the child processes output is read and appended to a string until
the stream is closed what usually happens when the program
terminates.
What I meant was just that the child process is really running and
writing "foo" and "baz" to the pipes 1 second before the output
can by noticed in the parent.
Bertram
···
Am Dienstag, 08. Sep 2009, 01:11:23 +0900 schrieb Eleanor McHugh:
Am Dienstag, 08. Sep 2009, 02:05:03 +0900 schrieb Bertram Scharpf:
What I meant was just that the child process is really running and
writing "foo" and "baz" to the pipes 1 second before the output
can by noticed in the parent.
In other words: the child processes don't block each other.
Is that ok?
(1..254).each do |i|
puts "pinging #{i}"
threads << Thread.new {
content = ""
IO.popen("ping 192.168.0.#{i}") do |io|
io.each { |l| content << l }
end
content
}
end
threads.each do |t|
t.join
#join is superfluous when using #value.
puts t.value
end
This doesn't behave different. My Ruby -v outputs this :
What exactly do you mean? What do you expect? If you refer to seeing the output of both ping commands sequentially: with the code you presented you always will get your output sequentially simply because you wait until all threads finish and then you'll iterate them and print the output of one thread at a time. Your ping commands will run in parallel.
Rather let's say that child processes won't block each other due to the underlying implementation of backtick. It is however still possible to introduce blocking behaviour by using operations in those children which would cause blocking, such as accessing a semaphore or other blocking system resource.
Am Dienstag, 08. Sep 2009, 02:05:03 +0900 schrieb Bertram Scharpf:
What I meant was just that the child process is really running and
writing "foo" and "baz" to the pipes 1 second before the output
can by noticed in the parent.
In other words: the child processes don't block each other.
Is that ok?
The first 4 threads or so are created almost instantly, and after that,
everything runs as if only one thing is executed at a time. I get the
same behaviour no matter how many times I try to run it.