Hello,
I was just trying out the thread pool code form Ruby Cookbook.
I created a script to test it out. The complete code is as follows:
···
-----------------------------------------------------------------------------
require 'thread'
class ThreadPool
def initialize(max_size)
@pool = []
@max_size = max_size
@pool_mutex = Mutex.new
@pool_cv = ConditionVariable.new
end
def dispatch(*args)
Thread.new do
# Wait for space in the pool.
@pool_mutex.synchronize do
while @pool.size >= @max_size
puts "Pool is full; waiting to run #{args.join(',')}…\n"
# Sleep until some other thread calls @pool_cv.signal.
@pool_cv.wait(@pool_mutex)
end
end
@pool << Thread.current
begin
yield(*args)
rescue => e
exception(self, e, *args)
ensure
@pool_mutex.synchronize do
# Remove the thread from the pool.
@pool.delete(Thread.current)
# Signal the next waiting thread that there's a space in the
pool.
@pool_cv.signal
end
end
end
end
def shutdown
@pool_mutex.synchronize { @pool_cv.wait(@pool_mutex) until
@pool.empty? }
end
def exception(thread, exception, *original_args)
# Subclass this method to handle an exception within a thread.
puts "Exception in thread #{thread}: #{exception}"
end
end
@work = []
def no_work_only_sleep
sleep(2)
@work << "Finished"
end
pool = ThreadPool.new(3)
10.times do |i|
pool.dispatch(i) do
puts "Job #{i} started"
no_work_only_sleep
end
end
pool.shutdown
puts @work.length
-------------------------------------------------------------------------
What I am doing here is creating 10 jobs and within a job calling a
method.
Inside the method I am inserting a string "Finished" to an array @work.
So for 10 jobs this array should contain 10 such strings.
The issue I have here is : For a thread pool of size 3 it gives an
output of 9 where it should have been 10.
For any other thread pool value it gives 10 as required.
Any reason as to why this is happening?
Thanks,
Sriram.
--
Posted via http://www.ruby-forum.com/.
Two things stand out:
1. as far as I can see you are not properly synchronizing access to @work.
2. IMHO the check of the current thread pool size should be done in
the calling thread in order to block it - not in the newly created
background thread.
Issue 2 is a bit tricky to understand but I believe you have created a
situation where background threads are started and waiting for room in
the pool to become available but the shutdown may get in the way so
you might terminate the pool although there are still threads waiting
to start their work.
Kind regards
robert
···
2010/3/18 Sriram Varahan <sriram.varahan@gmail.com>:
Hello,
I was just trying out the thread pool code form Ruby Cookbook\.
I created a script to test it out. The complete code is as follows:
-----------------------------------------------------------------------------
require 'thread'
class ThreadPool
def initialize(max_size)
@pool =
@max_size = max_size
@pool_mutex = Mutex.new
@pool_cv = ConditionVariable.new
end
def dispatch(*args)
Thread.new do
# Wait for space in the pool.
@pool_mutex.synchronize do
while @pool.size >= @max_size
puts "Pool is full; waiting to run #{args.join(',')}…\n"
# Sleep until some other thread calls @pool_cv.signal.
@pool_cv.wait(@pool_mutex)
end
end
@pool << Thread.current
begin
yield(*args)
rescue => e
exception(self, e, *args)
ensure
@pool_mutex.synchronize do
# Remove the thread from the pool.
@pool.delete(Thread.current)
# Signal the next waiting thread that there's a space in the
pool.
@pool_cv.signal
end
end
end
end
def shutdown
@pool_mutex.synchronize { @pool_cv.wait(@pool_mutex) until
@pool.empty? }
end
def exception(thread, exception, *original_args)
# Subclass this method to handle an exception within a thread.
puts "Exception in thread #{thread}: #{exception}"
end
end
@work =
def no_work_only_sleep
sleep(2)
@work << "Finished"
end
pool = ThreadPool.new(3)
10.times do |i|
pool.dispatch(i) do
puts "Job #{i} started"
no_work_only_sleep
end
end
pool.shutdown
puts @work.length
-------------------------------------------------------------------------
What I am doing here is creating 10 jobs and within a job calling a
method.
Inside the method I am inserting a string "Finished" to an array @work.
So for 10 jobs this array should contain 10 such strings.
The issue I have here is : For a thread pool of size 3 it gives an
output of 9 where it should have been 10.
For any other thread pool value it gives 10 as required.
Any reason as to why this is happening?
--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/