$stdin, thread blocking and DRb (for lack of a better subject :)

Greetings friends in Ruby-land,

I’ve crafted a Ruby application and its giving me a little trouble and I’m
hoping someone out there can give me some insight into the problem.

The app is composed of 3 threads (other than the primary thread, which waits
for the others to finish once its created them): the first thread is bound
to a code block which reads email via NET::POP3; the 2nd thread is that of a
simple DRb server; and the third thread is bound to a code block that
implements a simple command shell which reads lines from $stdin.

The application works wonderfully but w/the following problem: threads 1 and
2 block when the command shell thread reads from $stdin. Adjusting thread
priorities had no affect on the problem. When the command shell thread is
stopped all works as expected (email is read in the background and in
parallel w/the DRb server accepting requests from an test harness running as
a separate ruby session.) I’ve checked all the Ruby books and docs I have
and can’t find an explanation as to why this should be happening. My
suspicion is that because $stdin belongs to the ruby application main thread
and is being used by the command shell thread, that for some reason the
blocking read of $stdin is holding off the other threads. This speculation
is supported by the fact that entering a command to the shell allows all 3
threads to run in parallel until the command is completed and the next read
of $stdin occurs.

Any insight ya’ll can lend will be much appreciated.

Ken.

in ruby, if the process goes to sleep all threads stop because they are
non-native threads…

this program, however, seems to work in 1.6.8 or 1.8.0 so perhaps you have
another problem though?

~ > cat a.rb
#!/usr/bin/env ruby
require ‘thread’

begin
require ‘readline’
def prompt s
Readline::readline s, true
end
rescue
STDOUT.sync = true
def prompt
print s
gets
end
end

threads =
data = {}
mutex = Mutex.new

3.times do
threads <<
Thread.new(threads.size) do |tid|
loop{mutex.synchronize{(data[tid] ||= ) << [tid, Time.now]}; sleep 1}
end
end

loop do
cmd = prompt "select a thread #{ (0…threads.size).to_a.join ‘,’ } > "
case cmd
when /exit/io
exit
when /\d+/io
values = mutex.synchronize{data[cmd.to_i]}
(values || ).each{|val| p val}
end
end

~ > ruby a.rb
select a thread 0,1,2 > 1
[1, Tue Apr 27 15:46:46 MDT 2004]
[1, Tue Apr 27 15:46:47 MDT 2004]
select a thread 0,1,2 > 1
[1, Tue Apr 27 15:46:46 MDT 2004]
[1, Tue Apr 27 15:46:47 MDT 2004]
[1, Tue Apr 27 15:46:48 MDT 2004]
[1, Tue Apr 27 15:46:49 MDT 2004]
select a thread 0,1,2 > exit

so, how do you KNOW that all three threads are not running, if you run this
program you’ll see that they do seem to - even with the blocking read… i
think ruby tries very hard not to block the process…

-a

···

On Tue, 27 Apr 2004, Ken Hilton wrote:

Greetings friends in Ruby-land,

I’ve crafted a Ruby application and its giving me a little trouble and I’m
hoping someone out there can give me some insight into the problem.

The app is composed of 3 threads (other than the primary thread, which waits
for the others to finish once its created them): the first thread is bound
to a code block which reads email via NET::POP3; the 2nd thread is that of a
simple DRb server; and the third thread is bound to a code block that
implements a simple command shell which reads lines from $stdin.

The application works wonderfully but w/the following problem: threads 1 and
2 block when the command shell thread reads from $stdin. Adjusting thread
priorities had no affect on the problem. When the command shell thread is
stopped all works as expected (email is read in the background and in
parallel w/the DRb server accepting requests from an test harness running as
a separate ruby session.) I’ve checked all the Ruby books and docs I have
and can’t find an explanation as to why this should be happening. My
suspicion is that because $stdin belongs to the ruby application main thread
and is being used by the command shell thread, that for some reason the
blocking read of $stdin is holding off the other threads. This speculation
is supported by the fact that entering a command to the shell allows all 3
threads to run in parallel until the command is completed and the next read
of $stdin occurs.

Any insight ya’ll can lend will be much appreciated.

Ken.

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================

“Ara.T.Howard” ahoward@fattire.ngdc.noaa.gov wrote in message
news:Pine.LNX.4.44.0404271543580.5637-100000@fattire.ngdc.noaa.gov

Greetings friends in Ruby-land,

I’ve crafted a Ruby application and its giving me a little trouble and
I’m
hoping someone out there can give me some insight into the problem.

The app is composed of 3 threads (other than the primary thread, which
waits
for the others to finish once its created them): the first thread is
bound
to a code block which reads email via NET::POP3; the 2nd thread is that
of a
simple DRb server; and the third thread is bound to a code block that
implements a simple command shell which reads lines from $stdin.

The application works wonderfully but w/the following problem: threads 1
and
2 block when the command shell thread reads from $stdin. Adjusting
thread
priorities had no affect on the problem. When the command shell thread
is
stopped all works as expected (email is read in the background and in
parallel w/the DRb server accepting requests from an test harness
running as
a separate ruby session.) I’ve checked all the Ruby books and docs I
have
and can’t find an explanation as to why this should be happening. My
suspicion is that because $stdin belongs to the ruby application main
thread
and is being used by the command shell thread, that for some reason the
blocking read of $stdin is holding off the other threads. This
speculation
is supported by the fact that entering a command to the shell allows all
3
threads to run in parallel until the command is completed and the next
read
of $stdin occurs.

Any insight ya’ll can lend will be much appreciated.

Ken.

in ruby, if the process goes to sleep all threads stop because they are
non-native threads…

this program, however, seems to work in 1.6.8 or 1.8.0 so perhaps you have
another problem though?

~ > cat a.rb
#!/usr/bin/env ruby
require ‘thread’

begin
require ‘readline’
def prompt s
Readline::readline s, true
end
rescue
STDOUT.sync = true
def prompt
print s
gets
end
end

threads =
data = {}
mutex = Mutex.new

3.times do
threads <<
Thread.new(threads.size) do |tid|
loop{mutex.synchronize{(data[tid] ||= ) << [tid, Time.now]}; sleep
1}
end
end

loop do
cmd = prompt "select a thread #{ (0…threads.size).to_a.join ‘,’ } > "
case cmd
when /exit/io
exit
when /\d+/io
values = mutex.synchronize{data[cmd.to_i]}
(values || ).each{|val| p val}
end
end

~ > ruby a.rb
select a thread 0,1,2 > 1
[1, Tue Apr 27 15:46:46 MDT 2004]
[1, Tue Apr 27 15:46:47 MDT 2004]
select a thread 0,1,2 > 1
[1, Tue Apr 27 15:46:46 MDT 2004]
[1, Tue Apr 27 15:46:47 MDT 2004]
[1, Tue Apr 27 15:46:48 MDT 2004]
[1, Tue Apr 27 15:46:49 MDT 2004]
select a thread 0,1,2 > exit

so, how do you KNOW that all three threads are not running, if you run
this
program you’ll see that they do seem to - even with the blocking read…
i

···

On Tue, 27 Apr 2004, Ken Hilton wrote:
think ruby tries very hard not to block the process…

-a

============================================================================

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done

============================================================================

Thanks Ara,

I can’t try your program due to not having “readline.rb” I that part of the
Ruby 1.8.1 distribution? If not could you provide it to me? As for how (I
believe) I know that my threads are not running, they log events at a very
fine level to a shared log file (mutex protected.) As soon as the command
shell calls $stdin.gets the threads stop logging and the DRb thread stops
responding to remote method calls (they seem to be queued.) As soon as I
press [Enter] on my command shell and gets() returns the log file gets
updated, my POP3 mail fetch completes and DRb responds immediately. And, if
I simply comment out the gets() call and add a hard coded command string
that is processed repeatedly, all of the threads run in parallel. Hmmm…

Thanks Ara,

I can’t try your program due to not having “readline.rb” I that part of the
Ruby 1.8.1 distribution?

oops… should have been

‘rescue LoadError’

that way it’ll work w/o readline…

If not could you provide it to me? As for how (I believe) I know that my
threads are not running, they log events at a very fine level to a shared
log file (mutex protected.) As soon as the command shell calls $stdin.gets
the threads stop logging and the DRb thread stops responding to remote
method calls (they seem to be queued.) As soon as I press [Enter] on my
command shell and gets() returns the log file gets updated, my POP3 mail
fetch completes and DRb responds immediately. And, if I simply comment out
the gets() call and add a hard coded command string
that is processed repeatedly, all of the threads run in parallel. Hmmm…

sounds tough…

  • does the stdin read have the mutex while trying to gets?

-a

···

On Tue, 27 Apr 2004, Ken Hilton wrote:

===============================================================================

EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
PHONE :: 303.497.6469
ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
URL :: Solar-Terrestrial Physics Data | NCEI
TRY :: for l in ruby perl;do $l -e “print "\x3a\x2d\x29\x0a"”;done
===============================================================================