Problem: open-uri blocking redirection from http to https

ruby-talk
1

Hello,

I was working on a small script to verify the presence of an element through a list of URL, some of these URLs have a redirections from http to https, when the script crawled into them I got the following error:

/usr/lib/ruby/1.9.1/open-uri.rb:216:in `open_loop': redirection forbidden: http://beta.carsdirect.com/auto-loans/finance-app -> https://beta.carsdirect.com/auto-loans/finance-app (RuntimeError)

I understand that this is intentional, as per the comments on open-uri.rb

     # This test is intended to forbid a redirection from http://... to
     # file:///etc/passwd.
     # https to http redirect is also forbidden intentionally.
     # It avoids sending secure cookie or referer by non-secure HTTP protocol.
     # (RFC 2109 4.3.1, RFC 2965 3.3, RFC 2616 15.1.3)
     # However this is ad hoc. It should be extensible/configurable.

This mentions that "https to http" redirects are forbidden intentionally, but redirections from "http to https" are also blocked.

Is there a way to override this security check? currently I had to change the
following line in the library to allow "http to https" re-directions:

(/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~ uri2.scheme)

to

(/\A(?:http|ftp|https)\z/i =~ uri1.scheme && /\A(?:http|ftp|https)\z/i =~ uri2.scheme)

Thanks,
Xavi

2

Currently it is not configurable (as the comment says) except monkey patching.

Maybe open-uri should have some hooks.

···

2011/2/18 Xavier Del Castillo <xdelcastillo@archlinux.us>:

   # However this is ad hoc. It should be extensible/configurable.

This mentions that "https to http" redirects are forbidden intentionally,
but redirections from "http to https" are also blocked.

Is there a way to override this security check? currently I had to change
the
following line in the library to allow "http to https" re-directions:

(/\A(?:http|ftp)\z/i =~ uri1.scheme && /\A(?:http|ftp)\z/i =~ uri2.scheme)

--
Tanaka Akira