Net::HTTP Performance

ruby-talk
1

I know there was already another thread on this, but I didn't completely understand. I would love to re-read it but I accidentally deleted the emails. I remember something about changing a buffer size.

Can anyone please send me that thread or basically tell me what the fix was, because I could desperately use it.

Also, I'm not 100% sure this fix would apply to my situation, but basically I'm going out to a website and retrieving the HTML. Is that fix applicable here and would it improve speed?

Thanks for your help.

Thank You,
Ben Johnson
E: bjohnson@contuitive.com

2

Ben Johnson wrote:

I know there was already another thread on this, but I didn't completely understand. I would love to re-read it but I accidentally deleted the emails. I remember something about changing a buffer size.

Can anyone please send me that thread or basically tell me what the fix was, because I could desperately use it.

Also, I'm not 100% sure this fix would apply to my situation, but basically I'm going out to a website and retrieving the HTML. Is that fix applicable here and would it improve speed?

Thanks for your help.

Thank You,
Ben Johnson
E: bjohnson@contuitive.com

Yes, and yes. The fix is basically to go into your $RUBY_HOME/lib/ruby/1.8/net/protocol.rb, find the method called rbuf_fill and change the sysread to something bigger than 1024. Probably 8192 or 16384 would be good enough for your purposes.

···

--
  Ola Bini (http://ola-bini.blogspot.com)
  JvYAML, RbYAML, JRuby and Jatha contributor
  System Developer, Karolinska Institutet (http://www.ki.se)
  OLogix Consulting (http://www.ologix.com)

  "Yields falsehood when quined" yields falsehood when quined.

3

Its in the archives:

http://blade.nagaokaut.ac.jp/ruby/ruby-talk/index.shtml

And a google search works too:

http://groups.google.com/groups/search?q=net%3A%3Ahttp+performance+ruby

···

On Jul 17, 2006, at 1:01 AM, Ben Johnson wrote:

I know there was already another thread on this, but I didn't completely understand. I would love to re-read it but I accidentally deleted the emails.

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

4

Ola Bini wrote:

Yes, and yes. The fix is basically to go into your $RUBY_HOME/lib/ruby/1.8/net/protocol.rb, find the method called rbuf_fill and change the sysread to something bigger than 1024. Probably 8192 or 16384 would be good enough for your purposes.

Ruby being Ruby, you can also just reopen the class and redefine the method in question to your liking:

class ::Net::BufferedIO
  def rbuf_fill
    timeout(@read_timeout) {
      @rbuf << @io.sysread(8192)
    }
  end
end

But it's probably not a good idea to release code like this in a library to the public.