How to control thread number

ruby-talk
1

Hi all,

I copy the following script from Ruby programming language(bundling
with Ruby). If the size for pages (array size) is small,thread works
very well. If the array size is big (such 2000) the thread doesn't work
at or very slow. I wonder what is the best means to control thread
number based on this script, read a small trunk of array at a time using
each_slice method?

Thanks,

Li

##################### script#####################
pages = %w( www.rubycentral.com
            www.awl.com
            www.pragmaticprogrammer.com
           )

threads = []

for page in pages
  threads << Thread.new(page) { |myPage|

    h = Net::HTTP.new(myPage, 80)
    puts "Fetching: #{myPage}"
    resp, data = h.get('/', nil )
    puts "Got #{myPage}: #{resp.message}"
  }
end

···

--
Posted via http://www.ruby-forum.com/.

2

this is exactly what threadify does

cfp:~ > cat a.rb
require 'open-uri'

require 'rubygems'
require 'threadify'

# gem install threadify

···

On Sep 15, 2008, at 7:02 AM, Li Chen wrote:

Hi all,

I copy the following script from Ruby programming language(bundling
with Ruby). If the size for pages (array size) is small,thread works
very well. If the array size is big (such 2000) the thread doesn't work
at or very slow. I wonder what is the best means to control thread
number based on this script, read a small trunk of array at a time using
each_slice method?

Thanks,

Li

##################### script#####################
pages = %w( www.rubycentral.com
           www.awl.com
           www.pragmaticprogrammer.com
          )

threads =

for page in pages
threads << Thread.new(page) { |myPage|

   h = Net::HTTP.new(myPage, 80)
   puts "Fetching: #{myPage}"
   resp, data = h.get('/', nil )
   puts "Got #{myPage}: #{resp.message}"
}
end
--
Posted via http://www.ruby-forum.com/\.

#
# http://codeforpeople.com/lib/ruby/threadify/threadify-0.0.3/README

uris =
   %w(
     http://www.ruby-lang.com
     http://www.awl.com
     http://www.pragmaticprogrammer.com
   )

uris.threadify(2) do |uri|
   tid = Thread.current.object_id
   response = open(uri){|fd| fd.read}
   puts "#{ tid } : #{ uri } => #{ response[0,42].inspect }"
end

cfp:~ > ruby a.rb
1867720 : http://www.ruby-lang.com => "<html>\n<head>\n<title>Ruby Land</title>\n<me"
1864060 : http://www.awl.com => "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1"
1867720 : http://www.pragmaticprogrammer.com => "<!DOCTYPE html PUBLIC \"-//W3C//DTD XHTML 1"

this implementation is very short, you can read it so see one technique - which uses sized queues with a fixed number of producers and consumers

http://codeforpeople.com/lib/ruby/threadify/threadify-0.0.3/lib/threadify.rb

cheers.

a @ http://codeforpeople.com/
--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama

3

are you calling join on the threads?

···

--
Posted via http://www.ruby-forum.com/.

4

That's exactly what ThreadLimiter is designed for:

http://www.erikveen.dds.nl/threadlimiter/doc/index.html

gegroet,
Erik V.