Threaded Socket call blocks main thread

Hi all. A blocking IO issue is puzzling me, maybe someone can offer any insights.

I have written a plugin for Rails that does dns requests (to real-time blackhole lists). It works quite well. I am doing the dns requests (using Socket.gethostbyname) in separate threads. Unfortunately, when the call to Socket blocks, the main thread is blocking as well. And even though I have specified a parameter to thread.join, the script won't kill the blocked thread.

This is only a problem when no dns servers can be found or when they're terribly slow, but I must harden the plugin for these cases.

I have read posts on this list and it seems that any blocked IO such as what I am doing will block the main thread in Ruby. Is there a different way for me to kill the blocked thread and just move on? As this plugin is meant for web applications, a timely response is quite important. One or two seconds delay is fine, but no more.

Any help greatly appreciated.

require 'socket'

module DNSBL_Check
   $dnsbl_passed ||= []
   DNSBLS = %w{list.dsbl.org bl.spamcop.net sbl-xbl.spamhaus.org}

   private
   # Filter to check if the client is listed. This will be run before all requests.
   def dnsbl_check
     return true if $dnsbl_passed.include? request.remote_addr

     passed = true
     threads = []
     request.remote_addr =~ /(\d+).(\d+).(\d+).(\d+)/

     # Check the remote address against each dnsbl in a separate thread
     DNSBLS.each do |dnsbl|
       threads << Thread.new("#$4.#$3.#$2.#$1.#{dnsbl}") do |host|
         logger.warn("Checking DNSBL #{host}")
         if Socket.gethostbyname("#{host}")[3][0,2]=="\177\000"
           logger.info("#{request.remote_addr} found using DNSBL #{host}")
           passed = false
         end
       end
     end
     threads.each {|thread| thread.join(2)} # join threads, but use timeout to kill blocked ones

     # Add client ip to global passed cache if no dnsbls objected. else deny service.
     if passed
       $dnsbl_passed = $dnsbl_passed[0,49].unshift request.remote_addr
       logger.warn("#{request.remote_addr} added to DNSBL passed cache")
     else
       render :text => 'Access denied', :status => 403
       return false
     end
   end
end

···

--
Ga eens stemmen dit jaar! Nu SP! www.sp.nl

joost baaij wrote:

Hi all. A blocking IO issue is puzzling me, maybe someone can offer
any insights.

I have written a plugin for Rails that does dns requests (to real-
time blackhole lists). It works quite well. I am doing the dns
requests (using Socket.gethostbyname) in separate threads.
Unfortunately, when the call to Socket blocks, the main thread is
blocking as well. And even though I have specified a parameter to
thread.join, the script won't kill the blocked thread.

Obviously when you join the thread, it can block the main thread. But in my
experience with Ruby threads, if you don't join the thread, it dies. Which
leads a reasonable person to ask "what's the point of the threads?"

···

--
Paul Lutus
http://www.arachnoid.com

It seems very limited then. Just an easy way to parallelize (sp?) *some* tasks.

Foregoing threads, is there a way to continue a script that's waiting for a blocked Socket.gethostbyname call? Afaik only actual files can be opened non-blocking, right?

Thanks.

···

Op 21-okt-2006, om 8:10 heeft Paul Lutus het volgende geschreven:

Obviously when you join the thread, it can block the main thread. But in my
experience with Ruby threads, if you don't join the thread, it dies. Which
leads a reasonable person to ask "what's the point of the threads?"

http://shnoo.gr/articles/2005/12/13/resolv-replace

Truly a hidden ruby gem. I would have never found it on my own. Many thanks!!

···

Op 21-okt-2006, om 9:30 heeft snacktime het volgende geschreven:

http://shnoo.gr/articles/2005/12/13/resolv-replace