# Checking that an URL exists **URL:** \ **Category:** ruby-talk\ **Created:** [26 May 2004 12:55 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747 "2004-05-26T12:55:36Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![Imobach\_Gonzalez\_So2](https://avatars.discourse-cdn.com/v4/letter/i/858c86/32.png) [@Imobach\_Gonzalez\_So2](https://rubytalk.org/u/Imobach_Gonzalez_So2)\ **Post date:** [26 May 2004 12:55 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/1 "2004-05-26T12:55:36Z")
Hi all, Well, I want to make a little script that checks if the URLs in my bookmarks are OK (in other words, if they are online yet). I’ve been hanging around with URI classes and I’ve seen clearly a function that performs that check. I’ve tried using open-uri.rb but, if open method fails, the script fails. Any ideas? Thanks in advance. > **···** > > – > EuropeSwPatentFree > (o\_.’ Imobach González Sosa [imobachgs@softhome.net](mailto:imobachgs@softhome.net) > //\c{} [imobachgs@step.es](mailto:imobachgs@step.es)[a2419@dis.ulpgc.es](mailto:a2419@dis.ulpgc.es) > V\_\_)\_ [imodev@softhome.net](mailto:imodev@softhome.net) osoh en [jabber.at](http://jabber.at) y [jabber.org](http://jabber.org) > Usuario Linux #201634 > Gentoo Linux con núcleo 2.6.6 sobre Intel Pentium 4 ---
**Author:** ![Mohammad\_Khan3](https://avatars.discourse-cdn.com/v4/letter/m/ea666f/32.png) [@Mohammad\_Khan3](https://rubytalk.org/u/Mohammad_Khan3)\ **Post date:** [26 May 2004 14:09 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/2 "2004-05-26T14:09:05Z")
Try this: > **···** > > ################################################### > > #!/usr/bin/ruby > > require ‘net/http’ > > h = Net::HTTP.new(ARGV[0] || ‘[www.nonexist-ruby-lang.org](http://www.nonexist-ruby-lang.org)’, 80) > url = ARGV[1] || ‘/’ > > begin > resp, data = h.get(url, nil) { |a| } > puts "Valid URL" > rescue Net::ProtoRetriableError =\>detail > puts "Retriable URL" > rescue > puts "Not a Valid URL" > end > > #################################################### > > ----- Original Message ----- > From: “Imobach González Sosa” [imodev@softhome.net](mailto:imodev@softhome.net) > To: “ruby-talk ML” [ruby-talk@ruby-lang.org](mailto:ruby-talk@ruby-lang.org) > Sent: Wednesday, May 26, 2004 8:55 AM > Subject: Checking that an URL exists > > Hi all, > > Well, I want to make a little script that checks if the URLs in my bookmarks > are OK (in other words, if they are online yet). > > I’ve been hanging around with URI classes and I’ve seen clearly a function > that performs that check. > > I’ve tried using open-uri.rb but, if open method fails, the script fails. > > Any ideas? > > Thanks in advance. > > – > EuropeSwPatentFree > (o\_.’ Imobach González Sosa [imobachgs@softhome.net](mailto:imobachgs@softhome.net) > //\c{} [imobachgs@step.es](mailto:imobachgs@step.es)[a2419@dis.ulpgc.es](mailto:a2419@dis.ulpgc.es) > V\_\_)\_ [imodev@softhome.net](mailto:imodev@softhome.net) osoh en [jabber.at](http://jabber.at) y [jabber.org](http://jabber.org) > Usuario Linux #201634 > Gentoo Linux con núcleo 2.6.6 sobre Intel Pentium 4 ---
**Author:** ![Zach\_Dennis1](https://avatars.discourse-cdn.com/v4/letter/z/ea5d25/32.png) [@Zach\_Dennis1](https://rubytalk.org/u/Zach_Dennis1)\ **Post date:** [26 May 2004 14:14 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/3 "2004-05-26T14:14:51Z")
I am having trouble with my delete\_if block. prop\_array.delete\_if{ |p| (p=~/#{attr}/).to\_i \> 0; } and i have tried the longer version prop\_array.delete\_if{ |p| m = (p=~/#{attr}/).to\_i m \> 0; } and i have tried prop\_array.delete\_if{ |p| (p=~/#{attr}/) != nil } It deletes every element, even when m == 0 or m == nil. Any ideas? Thanks, Zach > **···** > > * * * > > Outgoing mail is certified Virus Free. > Checked by AVG anti-virus system ([http://www.grisoft.com](http://www.grisoft.com)). > Version: 6.0.684 / Virus Database: 446 - Release Date: 5/13/2004 ---
**Author:** ![Imobach\_Gonzalez\_So2](https://avatars.discourse-cdn.com/v4/letter/i/858c86/32.png) [@Imobach\_Gonzalez\_So2](https://rubytalk.org/u/Imobach_Gonzalez_So2)\ **Post date:** [26 May 2004 15:03 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/4 "2004-05-26T15:03:30Z")
Thank you… just before getting the mail i had just write a piece of code that could do the work… but your proposal looks better 😉 Thank you. > **···** > > El Miércoles, 26 de Mayo de 2004 15:09, Mohammad Khan escribió: > > > Try this: > > > > ################################################### > > > > #!/usr/bin/ruby > > > > require ‘net/http’ > > > > h = Net::HTTP.new(ARGV[0] || ‘[www.nonexist-ruby-lang.org](http://www.nonexist-ruby-lang.org)’, 80) > > url = ARGV[1] || ‘/’ > > > > begin > > resp, data = h.get(url, nil) { |a| } > > puts “Valid URL” > > rescue Net::ProtoRetriableError =\>detail > > puts “Retriable URL” > > rescue > > puts “Not a Valid URL” > > end > > > > #################################################### > > – > EuropeSwPatentFree > (o\_.’ Imobach González Sosa [imobachgs@softhome.net](mailto:imobachgs@softhome.net) > //\c{} imobachgs@step.es a2419@dis.ulpgc.es > V\_\_)\_ [imodev@softhome.net](mailto:imodev@softhome.net) osoh en jabber.at y [jabber.org](http://jabber.org) > Usuario Linux #201634 > Gentoo Linux con núcleo 2.6.6 sobre Intel Pentium 4 ---
**Author:** ![Mark\_Hubbart](https://avatars.discourse-cdn.com/v4/letter/m/b77776/32.png) [@Mark\_Hubbart](https://rubytalk.org/u/Mark_Hubbart)\ **Post date:** [26 May 2004 16:15 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/5 "2004-05-26T16:15:17Z")
> Try this: > > ################################################### > > #!/usr/bin/ruby > > require ‘net/http’ > > h = Net::HTTP.new(ARGV[0] || ‘[www.nonexist-ruby-lang.org](http://www.nonexist-ruby-lang.org)’, 80) > url = ARGV[1] || ‘/’ > > begin > resp, data = h.get(url, nil) { |a| } > puts “Valid URL” > rescue Net::ProtoRetriableError =\>detail > puts “Retriable URL” > rescue > puts “Not a Valid URL” > end > > #################################################### This is fine if you aren’t checking very many urls. If, however, you are going to be doing this very many times, you should use “head” instead, to reduce the webserver’s load (and increase your applications response time): resp = h.head(url, nil) Also, many invalid urls won’t raise errors, they just return error codes in the headers. You may want to check for that, too: h.head “/non-existant-page.html” =\> #\ h.head “/” =\> #\ h.head(“/”)[‘location’] =\> “[Ruby Programming Language](http://www.ruby-lang.org/en/)” h.head “/en/” =\> #\ h.head check the class of the header: Net::HTTPOK is a good url; Net::HTTPMovedPermanantly is what it sounds like, and you can treat it like a hash to look up the redirected location. Net::HTTPNotFound is a dead end. > **···** > > On May 26, 2004, at 7:09 AM, Mohammad Khan wrote: > > > ----- Original Message ----- > > From: “Imobach González Sosa” [imodev@softhome.net](mailto:imodev@softhome.net) > > To: “ruby-talk ML” [ruby-talk@ruby-lang.org](mailto:ruby-talk@ruby-lang.org) > > Sent: Wednesday, May 26, 2004 8:55 AM > > Subject: Checking that an URL exists > > > > Hi all, > > > > Well, I want to make a little script that checks if the URLs in my > > bookmarks > > are OK (in other words, if they are online yet). > > > > I’ve been hanging around with URI classes and I’ve seen clearly a > > function > > that performs that check. > > > > I’ve tried using open-uri.rb but, if open method fails, the script > > fails. > > > > Any ideas? > > > > Thanks in advance. > > > > – > > EuropeSwPatentFree > > (o\_.’ Imobach González Sosa [imobachgs@softhome.net](mailto:imobachgs@softhome.net) > > //\c{} imobachgs@step.es a2419@dis.ulpgc.es > > V\_\_)\_ [imodev@softhome.net](mailto:imodev@softhome.net) osoh en jabber.at y [jabber.org](http://jabber.org) > > Usuario Linux #201634 > > Gentoo Linux con núcleo 2.6.6 sobre Intel Pentium 4 ---
**Author:** ![Mohammad\_Khan3](https://avatars.discourse-cdn.com/v4/letter/m/ea666f/32.png) [@Mohammad\_Khan3](https://rubytalk.org/u/Mohammad_Khan3)\ **Post date:** [26 May 2004 14:41 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/6 "2004-05-26T14:41:38Z")
=~ Matches a regex against a string and return the offset of the start of the match or nil. Below example can illustrate this: > **···** > > ############################## > #!/usr/bin/ruby > > a = [‘at’, ‘atx’, ‘xatx’, ‘xat’, ‘xyz’] > > puts “\nBefore delete\_if array is : " > puts a.join(”\n") > > rgx = Regexp.new(/at/) > > puts “\n” > a.each { | x | puts “offset of 'at' in #{x} is #{rgx =~ x}” } > > ## delete\_if > > a.delete\_if { | x | rgx =~ x } > > puts “\nAfter delete\_if array is : " > puts a.join(”:") > > ############################# > > ----- Original Message ----- > From: “Zach Dennis” [zdennis@mktec.com](mailto:zdennis@mktec.com) > To: “ruby-talk ML” [ruby-talk@ruby-lang.org](mailto:ruby-talk@ruby-lang.org) > Sent: Wednesday, May 26, 2004 10:14 AM > Subject: delete\_if woes > > > I am having trouble with my delete\_if block. > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/).to\_i \> 0; } > > > > and i have tried the longer version > > > > prop\_array.delete\_if{ |p| > > m = (p=~/#{attr}/).to\_i > > m \> 0; } > > > > and i have tried > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/) != nil } > > > > It deletes every element, even when m == 0 or m == nil. Any ideas? Thanks, > > > > ## Zach > > > > Outgoing mail is certified Virus Free. > > Checked by AVG anti-virus system ([http://www.grisoft.com](http://www.grisoft.com)). > > Version: 6.0.684 / Virus Database: 446 - Release Date: 5/13/2004 ---
**Author:** ![David\_A\_Black3](https://avatars.discourse-cdn.com/v4/letter/d/6a8cbe/32.png) [@David\_A\_Black3](https://rubytalk.org/u/David_A_Black3)\ **Post date:** [26 May 2004 15:17 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/7 "2004-05-26T15:17:07Z")
Hi – > **···** > > On Wed, 26 May 2004, Zach Dennis wrote: > > > I am having trouble with my delete\_if block. > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/).to\_i \> 0; } > > > > and i have tried the longer version > > > > prop\_array.delete\_if{ |p| > > m = (p=~/#{attr}/).to\_i > > m \> 0; } > > > > and i have tried > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/) != nil } > > > > It deletes every element, even when m == 0 or m == nil. Any ideas? Thanks, > > I’m not sure exactly what you’re trying to do. The examples do > different things. Can you give sample before and after arrays? And > what’s in attr? > > David > > – > David A. Black > [dblack@wobblini.net](mailto:dblack@wobblini.net) ---
**Author:** ![Carlos](https://avatars.discourse-cdn.com/v4/letter/c/ecae2f/32.png) [@Carlos](https://rubytalk.org/u/Carlos)\ **Post date:** [26 May 2004 15:42 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/8 "2004-05-26T15:42:57Z")
[Zach Dennis [zdennis@mktec.com](mailto:zdennis@mktec.com), 2004-05-26 16.14 CEST] > I am having trouble with my delete\_if block. > > prop\_array.delete\_if{ |p| > (p=~/#{attr}/).to\_i \> 0; } […] > It deletes every element, even when m == 0 or m == nil. Any ideas? Thanks, Works here. Check the contents of prop\_array and attr. $ /usr/bin/ruby -ve ’ prop\_array= %w{ aax xaa } attr=“aa” result = prop\_array.delete\_if{ |p| (p=~/#{attr}/).to\_i \> 0; } p result ’ ruby 1.8.1 (2004-02-03) [i386-linux] [“aax”] $ ---
**Author:** ![Mohammad\_Khan3](https://avatars.discourse-cdn.com/v4/letter/m/ea666f/32.png) [@Mohammad\_Khan3](https://rubytalk.org/u/Mohammad_Khan3)\ **Post date:** [26 May 2004 15:52 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/9 "2004-05-26T15:52:48Z")
I think, I got you wrong. a = [‘at’, ‘atx’, ‘xatx’, ‘xat’, ‘xyz’] rgx = Regexp.new(/at/) a.delete\_if { | x | (rgx =~ x).to\_i \> 0 } #after this, my ‘a’ is # [“at”, “atx”, “xyz”] # which is logical and I dont see any odd. # what is #{attr} from your end? > **···** > > ----- Original Message ----- > From: “Zach Dennis” [zdennis@mktec.com](mailto:zdennis@mktec.com) > To: “ruby-talk ML” [ruby-talk@ruby-lang.org](mailto:ruby-talk@ruby-lang.org) > Sent: Wednesday, May 26, 2004 10:14 AM > Subject: delete\_if woes > > > I am having trouble with my delete\_if block. > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/).to\_i \> 0; } > > > > and i have tried the longer version > > > > prop\_array.delete\_if{ |p| > > m = (p=~/#{attr}/).to\_i > > m \> 0; } > > > > and i have tried > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/) != nil } > > > > It deletes every element, even when m == 0 or m == nil. Any ideas? Thanks, > > > > ## Zach > > > > Outgoing mail is certified Virus Free. > > Checked by AVG anti-virus system ([http://www.grisoft.com](http://www.grisoft.com)). > > Version: 6.0.684 / Virus Database: 446 - Release Date: 5/13/2004 ---
**Author:** ![Zach\_Dennis1](https://avatars.discourse-cdn.com/v4/letter/z/ea5d25/32.png) [@Zach\_Dennis1](https://rubytalk.org/u/Zach_Dennis1)\ **Post date:** [26 May 2004 16:52 UTC](https://rubytalk.org/t/checking-that-an-url-exists/11747/10 "2004-05-26T16:52:57Z")
Thanks for everyone’s response. My problem was that I needed to switch the ‘p’ and the ‘attr’ so it was: attr =~ /#{p}/ attr is something like: “Component.behavior” p would be something like: “behavior” Zach > **···** > > -----Original Message----- > From: David A. Black [[mailto:dblack@wobblini.net](mailto:dblack@wobblini.net)] > Sent: Wednesday, May 26, 2004 11:17 AM > To: ruby-talk ML > Subject: Re: delete\_if woes > > Hi – > > On Wed, 26 May 2004, Zach Dennis wrote: > > > I am having trouble with my delete\_if block. > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/).to\_i \> 0; } > > > > and i have tried the longer version > > > > prop\_array.delete\_if{ |p| > > m = (p=~/#{attr}/).to\_i > > m \> 0; } > > > > and i have tried > > > > prop\_array.delete\_if{ |p| > > (p=~/#{attr}/) != nil } > > > > It deletes every element, even when m == 0 or m == nil. Any ideas? Thanks, > > I’m not sure exactly what you’re trying to do. The examples do > different things. Can you give sample before and after arrays? And > what’s in attr? > > David > > – > David A. Black > [dblack@wobblini.net](mailto:dblack@wobblini.net) > > * * * > > Incoming mail is certified Virus Free. > Checked by AVG anti-virus system ([http://www.grisoft.com](http://www.grisoft.com)). > Version: 6.0.684 / Virus Database: 446 - Release Date: 5/13/2004 > > * * * > > Outgoing mail is certified Virus Free. > Checked by AVG anti-virus system ([http://www.grisoft.com](http://www.grisoft.com)). > Version: 6.0.684 / Virus Database: 446 - Release Date: 5/13/2004