HTTP.get problem

Ken_Kaplan · 8 July 2005 22:31

I'm new to ruby, and I've run into a problem while reading 'Programming
Ruby'. The 'Basic Input and Output' chapter has this example (I've added the
'puts' statements):

require 'net/http'

   h = Net::HTTP.new('www.pragmaticprogrammer.com', 80)
puts 'before get'
   resp, data = h.get('/index.html', nil)
puts 'after get'
   if resp.message == "OK"
     data.scan(/<img src="(.*?)"/) { |x| puts x }
   end

The above code hangs (but terminates ok with ctl-break) after 'before get',
and never reaches 'after get'. There is something up with my machine,
because a coworker can run this fine on his.

Things I've tried :

1. 'ping www.pragmaticprogrammer.com
<http://www.pragmaticprogrammer.com/> ' from the same command prompt. This
works fine.
2. try with other web sites. No difference
3. uninstalled, registry cleanup (CCleaner), reboot and
reinstalled ruby versions 182-14 and 182-15. No difference.
4. 'gem list ---remote' output gets to 'Updating Gem source
index for::/gems.rubyforge.org' and then hangs just like the above code.
5. Turned off firewall (ZoneAlarm) and anti-spyware (Microsoft).
No difference.
6. Added exception handling. No difference.

Any suggestions?

Ken

ruby --version

ruby 1.8.2 (2004-12-25) [i386-mswin32]

Gene_Tani · 9 July 2005 05:55

what does open-uri do?

daz · 9 July 2005 07:40

Ken Kaplan wrote:

Things I've tried : [...]

5. Turned off firewall (ZoneAlarm) - No difference.

I would have suspected that, first.
You *closed* ZA ? (Not recommended ever, while online, BTW)

You should have an entry under programs for:
Ruby interpreter 1,8,2,0
For this script you need: Allow connect (Internet)
I have "?Ask" Server (Local/Internet)

OK, you say it's not a firewall problem.

Any suggestions?

ruby 1.8.2 (2004-12-25) [i386-mswin32]

Here's a trace log of what should happen on your
version of Ruby.

http://www.d10.karoo.net/ruby/pptrace.txt

Try running this modified version of your script and
report which trace line number you reach.
If the last 10-20 lines of your trace vary from mine,
post them here and someone who understands this stuff
may be able to help.

···

#--------------------------------------------------------
$tr_line = 1
$tr_func = lambda do | ev, fi, li, ty, bi, cl |
  printf("%04d %14s:%4d %26s %-8s %s\n",
          $tr_line, File.basename(fi), li, ty, ev, cl)
  $tr_line += 1
end

STDOUT.sync=true
require 'net/http'

h = Net::HTTP.new('www.pragmaticprogrammer.com', 80)

set_trace_func($tr_func)
resp, data = h.get('/index.html', nil)
set_trace_func(nil)

puts 'after get'
if resp.message == "OK"
data.scan(/<img src="(.*?)"/) { |x| puts x }
end
#--------------------------------------------------------

daz

Gene_Tani · 9 July 2005 06:00

I think i was getting same thing, found the post:

http://groups-beta.google.com/group/comp.lang.ruby/browse_frm/thread/cbeb369da375a554/403af0ea585423ba?q=webfetcher+group:comp.lang.ruby&rnum=1&hl=en#403af0ea585423ba

anoth theory: my ISP's throttling/filtering me back.

Ken_Kaplan · 9 July 2005 17:25

daz,

Thank you for your help, kind sir.

daz wrote:

You *closed* ZA ? (Not recommended ever, while online, BTW)

Yes, it was painful to turn it off, but I'm getting desperate.

daz wrote:

Try running this modified version of your script and
report which trace line number you reach.

I reach line 272. Unfortunately, except for the file names we picked, my
trace file (http://factoad.com/ruby/pptrace_x.txt\) is identical to yours
(http://factoad.com/ruby/pptrace.txt\) up to line 0272, where mine hangs and
yours continues. (Here's the diff file:
http://factoad.com/ruby/tracediff.txt\. Here's the script:
http://factoad.com/ruby/io_test_trace.rb ).

It hangs when writing the get request string to a TCPsocket object.

Does anyone know why this might fail without raising an exception? Any
suggestions appreciated...

Ken

ruby 1.8.2 (2004-12-25) [i386-mswin32]
thread started with this email: http://factoad.com/ruby/email.txt

Bill_Pennington · 9 July 2005 18:08

I have not been following this thread closely so excuse me if I repeat advice from others. Since you are running windows and it looks like your are trying to access a raw socket maybe you have some kind of security setting that prevents raw socket access? I seem to recall some security update/recommended patch/tweak that prevented raw socket access from arbitrary programs.

···

On Jul 9, 2005, at 10:25 AM, Ken Kaplan wrote:

daz,

Thank you for your help, kind sir.

daz wrote:

You *closed* ZA ? (Not recommended ever, while online, BTW)

Yes, it was painful to turn it off, but I'm getting desperate.

daz wrote:

Try running this modified version of your script and
report which trace line number you reach.

I reach line 272. Unfortunately, except for the file names we picked, my
trace file (http://factoad.com/ruby/pptrace_x.txt\) is identical to yours
(http://factoad.com/ruby/pptrace.txt\) up to line 0272, where mine hangs and
yours continues. (Here's the diff file:
http://factoad.com/ruby/tracediff.txt\. Here's the script:
http://factoad.com/ruby/io_test_trace.rb ).

It hangs when writing the get request string to a TCPsocket object.

Does anyone know why this might fail without raising an exception? Any
suggestions appreciated...

Ken

ruby 1.8.2 (2004-12-25) [i386-mswin32]
thread started with this email: http://factoad.com/ruby/email.txt

- Bill

daz · 10 July 2005 04:40

Ken Kaplan wrote:

[...]

It hangs when writing the get request string to a TCPsocket object.

Does anyone know why this might fail without raising an exception?
Any suggestions appreciated...

I can't help, unfortunately.
I added -

h.set_debug_output $stdout # right after Net::HTTP.new

and got -

  opening connection to www.pragmaticprogrammer.com...
  opened
  <- "GET /index.html HTTP/1.1\r\nConnection: close\r\nHost: www.pragmaticprogrammer.com\r\n\r\n"
  -> "HTTP/1.1 200 OK\r\n"
  -> "Date: Sun, 10 Jul 2005 01:36:29 GMT\r\n"
  [...]

and (from your trace, thanks) you will be getting -

  opening connection to www.pragmaticprogrammer.com...
  opened
  [HANG]

(It's surprising that it doesn't time out, though.)

All I can suggest is that you try with a local server.
If you set this going in the background, you should
get a response from your browser at http://127.0.0.1/webrick :

<servlet.rb>

···

#-------------------------------------------------
require 'webrick'
include WEBrick

# Browser address: http://127.0.0.1/webrick

s = HTTPServer.new( :Port => 80 )

class HelloServlet < HTTPServlet::AbstractServlet
  def do_GET(req, res)
    # show request in response !
    res.body = req.to_s
    res['Content-Type'] = "text/plain"
  end
end
s.mount('/webrick', HelloServlet)

trap("INT") { s.shutdown }
STDOUT.sync=true
puts ' '*30 << '**********************************'
puts ' '*30 << '== Ctrl+Break to close server =='
puts ' '*30 << '**********************************'
s.start
#-------------------------------------------------

Then, if everything is OK of course, you could access
it from another Ruby session with your modified script:

#-------------------------------------------------
STDOUT.sync=true
require 'net/http'

h = Net::HTTP.new('127.0.0.1', 80)
h.set_debug_output STDOUT
resp, data = h.get('/webrick/xyz', nil)

puts '*'*30
if resp.message =~ /\AOK/
  print data
else
  puts 'ERROR ...'
  p resp.message
end
puts '*'*30
#-------------------------------------------------

It does seem like a problem you've got locally rather than
Ruby's Net library. Please post back if you find the cause
or suspect 'Net' is at fault.

Good luck,

daz

Ezra_Zygmuntowicz3 · 10 July 2005 05:03

I think you guys are having the same problem with windows and zone alarm that some people were having on the rails list. Zone alarm keeps ruby from working with sockets correctly. The only way they fixed this was by uninstalling zone alarm. Just turning it off didn't work but uninstalling it let ruby have its sockets back.

-Ezra

···

On Jul 9, 2005, at 9:40 PM, daz wrote:

Ken Kaplan wrote:

[...]

It hangs when writing the get request string to a TCPsocket object.

Does anyone know why this might fail without raising an exception?
Any suggestions appreciated...

I can't help, unfortunately.
I added -

  h.set_debug_output $stdout # right after Net::HTTP.new

and got -

  opening connection to www.pragmaticprogrammer.com...
  opened
  <- "GET /index.html HTTP/1.1\r\nConnection: close\r\nHost: www.pragmaticprogrammer.com\r\n\r\n"
  -> "HTTP/1.1 200 OK\r\n"
  -> "Date: Sun, 10 Jul 2005 01:36:29 GMT\r\n"
  [...]

and (from your trace, thanks) you will be getting -

  opening connection to www.pragmaticprogrammer.com...
  opened
  [HANG]

(It's surprising that it doesn't time out, though.)

All I can suggest is that you try with a local server.
If you set this going in the background, you should
get a response from your browser at http://127.0.0.1/webrick :

                  <servlet.rb>
#-------------------------------------------------
require 'webrick'
include WEBrick

# Browser address: http://127.0.0.1/webrick

s = HTTPServer.new( :Port => 80 )

class HelloServlet < HTTPServlet::AbstractServlet
  def do_GET(req, res)
    # show request in response !
    res.body = req.to_s
    res['Content-Type'] = "text/plain"
  end
end
s.mount('/webrick', HelloServlet)

trap("INT") { s.shutdown }
STDOUT.sync=true
puts ' '*30 << '**********************************'
puts ' '*30 << '== Ctrl+Break to close server =='
puts ' '*30 << '**********************************'
s.start
#-------------------------------------------------

Then, if everything is OK of course, you could access
it from another Ruby session with your modified script:

#-------------------------------------------------
STDOUT.sync=true
require 'net/http'

h = Net::HTTP.new('127.0.0.1', 80)
h.set_debug_output STDOUT
resp, data = h.get('/webrick/xyz', nil)

puts '*'*30
if resp.message =~ /\AOK/
  print data
else
  puts 'ERROR ...'
  p resp.message
end
puts '*'*30
#-------------------------------------------------

It does seem like a problem you've got locally rather than
Ruby's Net library. Please post back if you find the cause
or suspect 'Net' is at fault.

Good luck,

daz

-Ezra Zygmuntowicz
WebMaster
Yakima Herald-Republic Newspaper
ezra@yakima-herald.com
509-577-7732

daz · 10 July 2005 07:50

Ezra Zygmuntowicz wrote:

I think you guys are having the same problem with windows and zone
alarm that some people were having on the rails list. Zone alarm
keeps ruby from working with sockets correctly. The only way they
fixed this was by uninstalling zone alarm. Just turning it off didn't
work but uninstalling it let ruby have its sockets back.

-Ezra

Thanks for the tip, Ezra.

This Jeff Nadeau blog entry is very interesting:
http://www.jfnadeau.com/node/13
Describes the Ruby under Windows problem !! (Thanks, Jeff)

Anyone clued enough to contact him ?

Others with the problem:
http://forum.zonelabs.org/zonelabs/board/message?board.id=access&message.id=20205
http://www.ruby-talk.org/cgi-bin/scat.rb/ruby/ruby-talk/146253

Ken,

I'm using ZA 2.6.362 (went back to an old version for a different
reason a while back) on 98se and it's OK.

Which Windows and ZA do you have ?

daz

Ken_Kaplan · 11 July 2005 01:20

Problem fixed!

Ezra Zygmuntowicz wrote:

I think you guys are having the same problem with windows and zone
alarm that some people were having on the rails list. Zone alarm
keeps ruby from working with sockets correctly. The only way they
fixed this was by uninstalling zone alarm. Just turning it off didn't
work but uninstalling it let ruby have its sockets back.

-Ezra

Bingo. I uninstalled ZoneAlarm and now I can HTTP#put to my heart's content.
I can also run gem --remote.

daz wrote:

Which Windows and ZA do you have ?

I've got winxp sp2 with all the latest updates, and ZoneAlarm Suite 5-5-094,
with all their latest updates. The other computer that didn't have this
problem had the same winxp and ZoneAlarm updates, except that I had the paid
version and he had the free version.

I'll be trying it again with the free version, and contacting ZoneAlarm for
a workaround. I'll post again to this thread if I find one.

Thanks, folks.

Ken

Topic		Replies	Views
Problems with Net::HTTP.get ruby-talk	12	198	12 August 2005
Problems with Net/HTTP ruby-talk	2	81	1 February 2006
Net::HTTP.get_response hangs with particular url ruby-talk	2	118	4 April 2007
I can't establish network connections with Ruby ruby-talk	2	112	15 May 2005
Help with Ruby 1.9 - Net::HTTP calls hang on windows ruby-talk	0	115	12 April 2007

HTTP.get problem

Related topics