# Performance issues with net/http

**URL:** <https://rubytalk.org/t/performance-issues-with-net-http/28802>\
**Category:** ruby-talk\
**Created:** [10 July 2006 05:05 UTC](https://rubytalk.org/t/performance-issues-with-net-http/28802 "2006-07-10T05:05:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Luke\_Burton](https://avatars.discourse-cdn.com/v4/letter/l/ecb155/32.png) [@Luke\_Burton](https://rubytalk.org/u/Luke_Burton)\
**Post date:** [10 July 2006 05:05 UTC](https://rubytalk.org/t/performance-issues-with-net-http/28802/1 "2006-07-10T05:05:12Z")

</div>

It seems that net/http's implementation is extremely inefficient when  
it comes to dealing with large files.

I think this is something worth fixing in subsequent versions. It  
shouldn't be as bad as it is. I would also appreciate any hints or  
advice on working around the problem.

Below I have attached a test suite that illustrates the problem. I used  
WEBrick as the server.

"Host: localhost, port: 12000, request\_uri: /ten-meg.bin"  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;user system total real  
TCPSocket 0.030000 0.150000 0.180000 ( 0.468867)  
net/http 10.620000 8.630000 19.250000 ( 21.787785)  
LB net/http 10.870000 8.900000 19.770000 ( 22.259448)  
open-uri 16.400000 11.900000 28.300000 ( 39.834555)

As you can see, a raw TCPSocket is orders of magnitude faster than  
net/http and friends. However, I'm using read\_body and receiving the  
data in chunks, and I would have expected much better performance as a  
result. We're talking 20MB/s for TCPSocket versus 400KB/s for net/http.

What's happening here?

Any help appreciated.

Regards,

Luke.

#!/usr/bin/ruby

require 'net/http'  
require 'open-uri'  
require 'benchmark'  
require 'WEBrick'  
include WEBrick

uri = URI.parse("[http://localhost:12000/ten-meg.bin](http://localhost:12000/ten-meg.bin)")  
sourceFolder = "/tmp/"

Kernel.system("dd if=/dev/random of=/tmp/ten-meg.bin bs=1024  
count=10240")

port = 12000  
server = HTTPServer.new(:Port =\> port, :DocumentRoot =\> sourceFolder)  
# trap the signal for shutdown  
trap("INT"){ server.shutdown }  
pid = Kernel.fork {  
&nbsp;&nbsp;$stdout.reopen('/tmp/WEBrick.stdout')  
&nbsp;&nbsp;$stderr.reopen('/tmp/WEBrick.stderr')  
&nbsp;&nbsp;server.start  
}

at\_exit { Process.kill("INT", pid) }

Kernel.sleep 1

p "Host: #{uri.host}, port: #{uri.port}, request\_uri:  
#{uri.request\_uri}"

[Benchmark.bm](http://Benchmark.bm)(10) do |time|  
&nbsp;&nbsp;out = File.new("/tmp/tcp.tar.bz2", "w")  
&nbsp;&nbsp;time.report("TCPSocket") do  
&nbsp;&nbsp;&nbsp;&nbsp;s = TCPSocket.open uri.host, uri.port  
&nbsp;&nbsp;&nbsp;&nbsp;s.write "GET #{uri.request\_uri} HTTP/1.0\r\nHost:  
#{uri.host}\r\n\r\n"  
&nbsp;&nbsp;&nbsp;&nbsp;temp = s.read.split("\r\n\r\n", 2).last  
&nbsp;&nbsp;&nbsp;&nbsp;s.close  
&nbsp;&nbsp;&nbsp;&nbsp;out.write(temp)  
&nbsp;&nbsp;end  
&nbsp;&nbsp;out.close

&nbsp;&nbsp;out = File.new("/tmp/net.tar.bz2", "w")  
&nbsp;&nbsp;time.report("net/http") do  
&nbsp;&nbsp;&nbsp;&nbsp;Net::HTTP.start uri.host, uri.port do |http|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;http.request\_get(uri.request\_uri) do |response|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;response.read\_body do |segment|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out.write(segment)  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;end  
&nbsp;&nbsp;&nbsp;&nbsp;end  
&nbsp;&nbsp;end  
&nbsp;&nbsp;out.close

&nbsp;&nbsp;out = File.new("/tmp/luke.out", "w")  
&nbsp;&nbsp;time.report("LB net/http") do  
&nbsp;&nbsp;&nbsp;&nbsp;http = Net::HTTP.new(uri.host, uri.port)  
&nbsp;&nbsp;&nbsp;&nbsp;http.request\_get(uri.path) { |response|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;response.read\_body { |segment|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out.write(segment)  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;}  
&nbsp;&nbsp;&nbsp;&nbsp;}  
&nbsp;&nbsp;end  
&nbsp;&nbsp;out.close

&nbsp;&nbsp;out = File.new("/tmp/uri.tar.bz2", "w")  
&nbsp;&nbsp;time.report("open-uri") do  
&nbsp;&nbsp;&nbsp;&nbsp;uri.open do |x|  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out.write(x.read)  
&nbsp;&nbsp;&nbsp;&nbsp;end  
&nbsp;&nbsp;end  
&nbsp;&nbsp;out.close  
end