Open-uri + OpenSSL

ruby-talk
1

Hello,

I am usring open-uri to open an https:// link and when it tries to read
it, I get the 'connect' : certificate verify failed error. How can I
bypass this SSL verification?

Thanks,
M

···

--
Posted via http://www.ruby-forum.com/.

2

Set the verify_mode to OpenSSL::SSL::VERIFY_NONE

eg:

http = Net::HTTP.new(host,port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE

···

On Oct 27, 1:35 am, Matthew Lagace <matthewlag...@gmail.com> wrote:

Hello,

I am usring open-uri to open an https:// link and when it tries to read
it, I get the 'connect' : certificate verify failed error. How can I
bypass this SSL verification?

Thanks,
M
--
Posted viahttp://www.ruby-forum.com/.

3

Although the ticket remains as open in the page, the feature has been
added at least for ruby-1.9.3-p125.

You can pass :ssl_verify_mode => OpenSSL::SSL::VERIFY_NONE as an option
to the open method.

Best Regards

···

--
Posted via http://www.ruby-forum.com/.

4

As Fernando points out, this feature is now available, at least post
1.9.3-p327. Here's how I'm using it:

  request_uri=URI.parse('myurl')
  request_uri.query=URI.encode_www_form params

  output = open(request_uri, {ssl_verify_mode:
OpenSSL::SSL::VERIFY_NONE})
  obj = JSON.parse output.readlines.join("")

sameer.

···

--
Posted via http://www.ruby-forum.com/.

5

Ok when I do that, it says:

ssl value changed, but session already started

dusty wrote:

···

On Oct 27, 1:35 am, Matthew Lagace <matthewlag...@gmail.com> wrote:

Hello,

I am usring open-uri to open an https:// link and when it tries to read
it, I get the 'connect' : certificate verify failed error. How can I
bypass this SSL verification?

Thanks,
M
--
Posted viahttp://www.ruby-forum.com/.

Set the verify_mode to OpenSSL::SSL::VERIFY_NONE

eg:

http = Net::HTTP.new(host,port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE

--
Posted via http://www.ruby-forum.com/\.

6

Sorry, I guess you can't do it with open-uri. Here is a patch:

add this ssl_verify option to the top of the file.

FROM:

module OpenURI
  Options = {
    :proxy => true,
    :progress_proc => true,
    :content_length_proc => true,
    :http_basic_authentication => true,
  }

TO:

module OpenURI
  Options = {
    :proxy => true,
    :progress_proc => true,
    :content_length_proc => true,
    :http_basic_authentication => true,
    :ssl_verify => true
  }

Change the part where it enables verification

FROM:

    if target.class == URI::HTTPS
      require 'net/https'
      http.use_ssl = true
      http.enable_post_connection_check = true
      http.verify_mode = OpenSSL::SSL::VERIFY_PEER
      store = OpenSSL::X509::Store.new
      store.set_default_paths
      http.cert_store = store
    end

TO:
    if target.class == URI::HTTPS
      require 'net/https'
      http.use_ssl = true
      http.enable_post_connection_check = true
      if options[:ssl_verify] == false
        http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      else
        http.verify_mode = OpenSSL::SSL::VERIFY_PEER
      end
      store = OpenSSL::X509::Store.new
      store.set_default_paths
      http.cert_store = store
    end

run it like this:

open("https://someurl", :ssl_verify => false) {|f|
  print f.read
}

···

On Oct 27, 12:26 pm, Matthew Lagace <matthewlag...@gmail.com> wrote:

Ok when I do that, it says:

ssl value changed, but session already started

dusty wrote:
> On Oct 27, 1:35 am, Matthew Lagace <matthewlag...@gmail.com> wrote:
>> Hello,

>> I am usring open-uri to open an https:// link and when it tries to read
>> it, I get the 'connect' : certificate verify failed error. How can I
>> bypass this SSL verification?

>> Thanks,
>> M
>> --
>> Posted viahttp://www.ruby-forum.com/.

> Set the verify_mode to OpenSSL::SSL::VERIFY_NONE

> eg:

> http = Net::HTTP.new(host,port)
> http.use_ssl = true
> http.verify_mode = OpenSSL::SSL::VERIFY_NONE

--
Posted viahttp://www.ruby-forum.com/.

7

Sorry, this all goes in open-uri.rb in your ruby base dir, eg:

/usr/lib/ruby/1.8/open-uri.rb
or
/opt/local/lib/ruby/1.8/open-uri.rb

or wherever it may be on your distro.

···

On Nov 5, 8:51 pm, dusty <dusty.do...@gmail.com> wrote:

On Oct 27, 12:26 pm, Matthew Lagace <matthewlag...@gmail.com> wrote:

> Ok when I do that, it says:

> ssl value changed, but session already started

> dusty wrote:
> > On Oct 27, 1:35 am, Matthew Lagace <matthewlag...@gmail.com> wrote:
> >> Hello,

> >> I am usring open-uri to open an https:// link and when it tries to read
> >> it, I get the 'connect' : certificate verify failed error. How can I
> >> bypass this SSL verification?

> >> Thanks,
> >> M
> >> --
> >> Posted viahttp://www.ruby-forum.com/.

> > Set the verify_mode to OpenSSL::SSL::VERIFY_NONE

> > eg:

> > http = Net::HTTP.new(host,port)
> > http.use_ssl = true
> > http.verify_mode = OpenSSL::SSL::VERIFY_NONE

> --
> Posted viahttp://www.ruby-forum.com/.

Sorry, I guess you can't do it with open-uri. Here is a patch:

add this ssl_verify option to the top of the file.

FROM:

module OpenURI
  Options = {
    :proxy => true,
    :progress_proc => true,
    :content_length_proc => true,
    :http_basic_authentication => true,
  }

TO:

module OpenURI
  Options = {
    :proxy => true,
    :progress_proc => true,
    :content_length_proc => true,
    :http_basic_authentication => true,
    :ssl_verify => true
  }

Change the part where it enables verification

FROM:

    if target.class == URI::HTTPS
      require 'net/https'
      http.use_ssl = true
      http.enable_post_connection_check = true
      http.verify_mode = OpenSSL::SSL::VERIFY_PEER
      store = OpenSSL::X509::Store.new
      store.set_default_paths
      http.cert_store = store
    end

TO:
    if target.class == URI::HTTPS
      require 'net/https'
      http.use_ssl = true
      http.enable_post_connection_check = true
      if options[:ssl_verify] == false
        http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      else
        http.verify_mode = OpenSSL::SSL::VERIFY_PEER
      end
      store = OpenSSL::X509::Store.new
      store.set_default_paths
      http.cert_store = store
    end

run it like this:

open("https://someurl", :ssl_verify => false) {|f|
  print f.read

}

8

dusty wrote:

Ok when I do that, it says:
ssl value changed, but session already started
dusty wrote:

Hello,
I am usring open-uri to open an https:// link and when it tries to read
it, I get the 'connect' : certificate verify failed error. How can I
bypass this SSL verification?
Thanks,
M
--
Posted viahttp://www.ruby-forum.com/.

Set the verify_mode to OpenSSL::SSL::VERIFY_NONE
eg:
http = Net::HTTP.new(host,port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_NONE

--
Posted viahttp://www.ruby-forum.com/.

Sorry, I guess you can't do it with open-uri. Here is a patch:

add this ssl_verify option to the top of the file.

FROM:

module OpenURI
  Options = {
    :proxy => true,
    :progress_proc => true,
    :content_length_proc => true,
    :http_basic_authentication => true,
  }

TO:

module OpenURI
  Options = {
    :proxy => true,
    :progress_proc => true,
    :content_length_proc => true,
    :http_basic_authentication => true,
    :ssl_verify => true
  }

Change the part where it enables verification

FROM:

    if target.class == URI::HTTPS
      require 'net/https'
      http.use_ssl = true
      http.enable_post_connection_check = true
      http.verify_mode = OpenSSL::SSL::VERIFY_PEER
      store = OpenSSL::X509::Store.new
      store.set_default_paths
      http.cert_store = store
    end

TO:
    if target.class == URI::HTTPS
      require 'net/https'
      http.use_ssl = true
      http.enable_post_connection_check = true
      if options[:ssl_verify] == false
        http.verify_mode = OpenSSL::SSL::VERIFY_NONE
      else
        http.verify_mode = OpenSSL::SSL::VERIFY_PEER
      end
      store = OpenSSL::X509::Store.new
      store.set_default_paths
      http.cert_store = store
    end

run it like this:

open("https://someurl", :ssl_verify => false) {|f|
  print f.read

}

Sorry, this all goes in open-uri.rb in your ruby base dir, eg:

/usr/lib/ruby/1.8/open-uri.rb
or
/opt/local/lib/ruby/1.8/open-uri.rb

or wherever it may be on your distro.

The nice thing about Ruby is that if you don't want to modify your
system files (for example I don't like to do it because it's quite hard
to track later), then you can simply patch the modules/classes in
question on the fly, at the beginning of your application. And possibly
file a bug report (:

However, I guess the verification-enabling code would be more versatile
this way:

       if options[:ssl_verify]
         http.verify_mode = OpenSSL::SSL::VERIFY_PEER
       else
         http.verify_mode = OpenSSL::SSL::VERIFY_NONE
       end

mortee

···

On Nov 5, 8:51 pm, dusty <dusty.do...@gmail.com> wrote:

On Oct 27, 12:26 pm, Matthew Lagace <matthewlag...@gmail.com> wrote:

On Oct 27, 1:35 am, Matthew Lagace <matthewlag...@gmail.com> wrote:

9

Great thanks guys!

···

--
Posted via http://www.ruby-forum.com/.

10

Good idea. I submitted a patch to rubyforge. This might be useful
and simple enough to add.

http://rubyforge.org/tracker/?group_id=426&atid=1698&func=detail&aid=15390

···

On Nov 5, 11:34 pm, mortee <mortee.li...@kavemalna.hu> wrote:

dusty wrote:
> On Nov 5, 8:51 pm, dusty <dusty.do...@gmail.com> wrote:
>> On Oct 27, 12:26 pm, Matthew Lagace <matthewlag...@gmail.com> wrote:

>>> Ok when I do that, it says:
>>> ssl value changed, but session already started
>>> dusty wrote:
>>>> On Oct 27, 1:35 am, Matthew Lagace <matthewlag...@gmail.com> wrote:
>>>>> Hello,
>>>>> I am usring open-uri to open an https:// link and when it tries to read
>>>>> it, I get the 'connect' : certificate verify failed error. How can I
>>>>> bypass this SSL verification?
>>>>> Thanks,
>>>>> M
>>>>> --
>>>>> Posted viahttp://www.ruby-forum.com/.
>>>> Set the verify_mode to OpenSSL::SSL::VERIFY_NONE
>>>> eg:
>>>> http = Net::HTTP.new(host,port)
>>>> http.use_ssl = true
>>>> http.verify_mode = OpenSSL::SSL::VERIFY_NONE
>>> --
>>> Posted viahttp://www.ruby-forum.com/.
>> Sorry, I guess you can't do it with open-uri. Here is a patch:

>> add this ssl_verify option to the top of the file.

>> FROM:

>> module OpenURI
>> Options = {
>> :proxy => true,
>> :progress_proc => true,
>> :content_length_proc => true,
>> :http_basic_authentication => true,
>> }

>> TO:

>> module OpenURI
>> Options = {
>> :proxy => true,
>> :progress_proc => true,
>> :content_length_proc => true,
>> :http_basic_authentication => true,
>> :ssl_verify => true
>> }

>> Change the part where it enables verification

>> FROM:

>> if target.class == URI::HTTPS
>> require 'net/https'
>> http.use_ssl = true
>> http.enable_post_connection_check = true
>> http.verify_mode = OpenSSL::SSL::VERIFY_PEER
>> store = OpenSSL::X509::Store.new
>> store.set_default_paths
>> http.cert_store = store
>> end

>> TO:
>> if target.class == URI::HTTPS
>> require 'net/https'
>> http.use_ssl = true
>> http.enable_post_connection_check = true
>> if options[:ssl_verify] == false
>> http.verify_mode = OpenSSL::SSL::VERIFY_NONE
>> else
>> http.verify_mode = OpenSSL::SSL::VERIFY_PEER
>> end
>> store = OpenSSL::X509::Store.new
>> store.set_default_paths
>> http.cert_store = store
>> end

>> run it like this:

>> open("https://someurl", :ssl_verify => false) {|f|
>> print f.read

>> }

> Sorry, this all goes in open-uri.rb in your ruby base dir, eg:

> /usr/lib/ruby/1.8/open-uri.rb
> or
> /opt/local/lib/ruby/1.8/open-uri.rb

> or wherever it may be on your distro.

The nice thing about Ruby is that if you don't want to modify your
system files (for example I don't like to do it because it's quite hard
to track later), then you can simply patch the modules/classes in
question on the fly, at the beginning of your application. And possibly
file a bug report (:

However, I guess the verification-enabling code would be more versatile
this way:

       if options[:ssl_verify]
         http.verify_mode = OpenSSL::SSL::VERIFY_PEER
       else
         http.verify_mode = OpenSSL::SSL::VERIFY_NONE
       end

mortee

11

Any news about the status of this?

···

--
Posted via http://www.ruby-forum.com/.

12

i too am waiting for this patch. appreciate early response.

Seede

···

On Jun 2, 7:01 am, Marc Heiler <sheve...@linuxmail.org> wrote:

Any news about the status of this?
--
Posted viahttp://www.ruby-forum.com/.

13

i tried to put the patch into my open-uri.rb and it failed with a new
error
irb(main):007:0* open("https://www.interactivebrokers.com/Universal/
servlet/FlexStatementService.GetStatement?
t=1437758&q=1126698&v=2",:ssl_verify => true){|f|puts f}
NoMethodError: undefined method `enable_post_connection_check=' for
#<Net::HTTP www.interactivebrokers.com:443 open=false>
        from e:/ruby/lib/ruby/1.8/open-uri.rb:242:in `open_http'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:643:in `buffer_open'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:170:in `open_loop'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:168:in `catch'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:168:in `open_loop'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:138:in `open_uri'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:545:in `open'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:30:in `open'
        from (irb):7

···

On Jun 2, 7:01 am, Marc Heiler <sheve...@linuxmail.org> wrote:

Any news about the status of this?
--
Posted viahttp://www.ruby-forum.com/.

14

Junkone wrote:

Any news about the status of this?
--
Posted viahttp://www.ruby-forum.com/.

i tried to put the patch into my open-uri.rb and it failed with a new
error
irb(main):007:0* open("https://www.interactivebrokers.com/Universal/
servlet/FlexStatementService.GetStatement?
t=1437758&q=1126698&v=2",:ssl_verify => true){|f|puts f}
NoMethodError: undefined method `enable_post_connection_check=' for
#<Net::HTTP www.interactivebrokers.com:443 open=false>
        from e:/ruby/lib/ruby/1.8/open-uri.rb:242:in `open_http'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:643:in `buffer_open'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:170:in `open_loop'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:168:in `catch'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:168:in `open_loop'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:138:in `open_uri'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:545:in `open'
        from e:/ruby/lib/ruby/1.8/open-uri.rb:30:in `open'
        from (irb):7

So basically if you are using ruby 1.8.6 (for me I'm using it on
BackTrack3) you simply omit the 'http.enable_post_connection_check =
true' from the code mentioned above and also remove
'sock.post_connection_check(target_host)'
that's it! I am using the open-uri just fine now to push thru multiple
URLs from an input file out to my local proxy in order to built a site
map via BurpSuite.

···

On Jun 2, 7:01�am, Marc Heiler <sheve...@linuxmail.org> wrote:

--
Posted via http://www.ruby-forum.com/\.