Net/ftp is cpu bound on fast internet connections


(jheard) #1

I was trying to run multiple ftp.putbinaryfile jobs and noticed that if
I run more than say 2 threads on my PIII 933, My CPU usage hits 100% and
my IO stays about the same, even though I am nowhere close to my
theorectical limit. Here’s the original code.

1.upto(5) {|server_suffix|
puts "starting send to DL360#{server_suffix}"
threads[server_suffix] = Thread.new {
ftp[server_suffix].putbinaryfile ‘100recs.tar.gz’,
‘100recs.tar.gz’

     telnet[server_suffix].cmd 'tar -xzf 100recs.tar.gz'

}
}
threads[1…5].each {|thread| thread.join}

My connection should actually be able to handle about 20 of these
threads at a time. What I noticed was that of course ftp.rb’s routines
are block oriented and pass everything through yield to a block for
processing. I was wondering if it would be good for me or someone to
add non block-oriented functions that perhaps even memory map and or
buffer files where appropriate to save cpu time.