Problem with trivial regular expression

ruby-talk
1

hi ,sorry for my english.

I am trying to remember the use of regular expressions and i have a
problem with this :

i had a text with diferents strings, for example, url , like this :

fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja

i want to extract the diferents url but i try with :
/(http\:\/\/.+com)/ it returns a long match:

   1. http://www.marca.comjafosjodfahttp://www.as.com

how can i group this in 2 diferents matchs? example :
1- http://www.marca.com
2- http://www.as.com

thanks

···

--
Posted via http://www.ruby-forum.com/.

2

What you are missing is the non-greedy modifier (?) for the +:

irb(main):001:0> s = "
fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja"
=> " fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja"
irb(main):003:0> s.scan(/http\:\/\/.+?\.com/)
=> ["http://www.marca.com", "http://www.as.com"]

(I also added an extra \. before com, to match a ".com" and not "com"
only). Then, scan helps you going through the full string retrieving
matches.

Hope this helps,

Jesus.

···

On Mon, Dec 14, 2009 at 11:06 AM, David Villa <villa.hxc@gmail.com> wrote:

hi ,sorry for my english.

I am trying to remember the use of regular expressions and i have a
problem with this :

i had a text with diferents strings, for example, url , like this :

fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja

i want to extract the diferents url but i try with :
/(http\:\/\/.+com)/ it returns a long match:

1. http://www.marca.comjafosjodfahttp://www.as.com

how can i group this in 2 diferents matchs? example :
1- http://www.marca.com
2- http://www.as.com

thanks
--
Posted via http://www.ruby-forum.com/\.

3

s =

"fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja"
=> "fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja"

s.scan( %r{http://.+?.com} )

=> ["http://www.marca.com", "http://www.as.com"]

I use scan because we want multiple results.

For the Regexp,
in ".+?", you make it ungreedy
The %r{} let you write / without escaping
I also add a "\." to ensure there is a point before "com"

Enjoy

···

2009/12/14 David Villa <villa.hxc@gmail.com>

hi ,sorry for my english.

I am trying to remember the use of regular expressions and i have a
problem with this :

i had a text with diferents strings, for example, url , like this :

fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja

i want to extract the diferents url but i try with :
/(http\:\/\/.+com)/ it returns a long match:

  1. http://www.marca.comjafosjodfahttp://www.as.com

how can i group this in 2 diferents matchs? example :
1- http://www.marca.com
2- http://www.as.com

thanks
--
Posted via http://www.ruby-forum.com/\.

4

fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja

Any particular context? Or is it actually that random?

i want to extract the diferents url but i try with :
/(http\:\/\/.+com)/ it returns a long match:

   1. http://www.marca.comjafosjodfahttp://www.as.com

If you think about it, that is still a valid URL. You're trying to limit it
not to URLs, but only to http:// followed by a domain, and then only a domain
ending in .com -- there are MANY urls that this will break.

If you're OK with that, the basic problem is that . is going to match as much
as it possibly can (greedy), and it matches any character. The simple solution
is to make it match as few characters as it can (miserly). You do that by
putting a question mark after the + or *:

/(http\:\/\/.+?com)/

But again, that's not matching .com, that's matching anything ending in com.
For example, on this URL:

it will only capture http://www.broadcom. So there's an easy solution -- add
an escaped dot:

/(http\:\/\/.+?\.com)/

That's as much as I want to do with it. I'm guessing what you're trying to do
is auto-linkify URLs in forum posts, or something like that -- some problem
that's been solved a million times before, and better, so you should look for
those solutions. But I won't assume that applies to you...

By the way, if you don't already know:

···

On Monday 14 December 2009 04:06:08 am David Villa wrote:

5

We are quite according in our posts :slight_smile:
This should be the Ruby way, very clear this time !

···

2009/12/14 Jesús Gabriel y Galán <jgabrielygalan@gmail.com>

On Mon, Dec 14, 2009 at 11:06 AM, David Villa <villa.hxc@gmail.com> wrote:
>
> hi ,sorry for my english.
>
> I am trying to remember the use of regular expressions and i have a
> problem with this :
>
> i had a text with diferents strings, for example, url , like this :
>
> fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja
>
> i want to extract the diferents url but i try with :
> /(http\:\/\/.+com)/ it returns a long match:
>
>
>
> 1. http://www.marca.comjafosjodfahttp://www.as.com
>
>
> how can i group this in 2 diferents matchs? example :
> 1- http://www.marca.com
> 2- http://www.as.com
>
>
> thanks
> --
> Posted via http://www.ruby-forum.com/\.
>
>

What you are missing is the non-greedy modifier (?) for the +:

irb(main):001:0> s = "
fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja"
=> "
fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja"
irb(main):003:0> s.scan(/http\:\/\/.+?\.com/)
=> ["http://www.marca.com", "http://www.as.com"]

(I also added an extra \. before com, to match a ".com" and not "com"
only). Then, scan helps you going through the full string retrieving
matches.

Hope this helps,

Jesus.

6

David Masover wrote:

   1. http://www.marca.comjafosjodfahttp://www.as.com

If you think about it, that is still a valid URL.

That's arguable, because of the colon. RFC 1738:

   URL schemes that involve the direct use
   of an IP-based protocol to a specified host on the Internet use a
   common syntax for the scheme-specific data:

        //<user>:<password>@<host>:<port>/<url-path>

...
    port
        The port number to connect to. Most schemes designate
        protocols that have a default port number. Another port number
        may optionally be supplied, in decimal, separated from the
        host by a colon. If the port is omitted, the colon is as well.

However, it says "is" rather than "MUST BE".

···

--
Posted via http://www.ruby-forum.com/\.

7

Thanks for all answer,

i used rubular.com for test, thanks however.

I posted a random string and i understand it better, but the real
question and string is :

"\"18%7Chttp%3A%2F%2Fv14.lscache3.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D18%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D4C33C8CB5787DBA4D111A2E66BFFACCC2E95E0A7.D302F87BF11A4BD47631704E858C762325913921%26factor%3D1.25%26id%3Dcf9829e68818de48%2C34%7Chttp%3A%2F%2Fv7.lscache8.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D34%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D158633D23CDC66CC40D171EC8CB48A84B4BCB2
23.86F6779A36E14684B367BCCF91588556A871C421%26factor%3D1.25%26id%3Dcf9829e68818de48%2C5%7Chttp%3A%2F%2Fv16.lscache8.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D5%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D91FAC179A7C02DD662942CCEB71D8BE05BD7B5D3.BC4BFC97F8BA02519CC817D2541E5D0548BC2C7C%26factor%3D1.25%26id%3Dcf9829e68818de48\"

here, are some urls, one starts with : http.... and end with %2C3 :

http%3A%2F%2Fv14.lscache3.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D18%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D4C33C8CB5787DBA4D111A2E66BFFACCC2E95E0A7.D302F87BF11A4BD47631704E858C762325913921%26factor%3D1.25%26id%3Dcf9829e68818de48%2C3

and other, following this, with star with : http...and end with %2C5.

http%3A%2F%2Fv7.lscache8.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D34%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D158633D23CDC66CC40D171EC8CB48A84B4BCB223.86F6779A36E14684B367BCCF91588556A871C421%26factor%3D1.25%26id%3Dcf9829e68818de48%2C5

So, when i match the frist, no problem, but when i try to match the
second, it matchs all the submatch,the first and the second :

Match captures:

   1.
http%3A%2F%2Fv14.lscache3.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D18%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D4C33C8CB5787DBA4D111A2E66BFFACCC2E95E0A7.D302F87BF11A4BD47631704E858C762325913921%26factor%3D1.25%26id%3Dcf9829e68818de48%2C34%7Chttp%3A%2F%2Fv7.lscache8.c.youtube.com%2Fvideoplayback%3Fip%3D0.0.0.0%26sparams%3Did%252Cexpire%252Cip%252Cipbits%252Citag%252Calgorithm%252Cburst%252Cfactor%26fexp%3D900034%252C902305%26algorithm%3Dthrottle-factor%26itag%3D34%26ipbits%3D0%26burst%3D40%26sver%3D3%26expire%3D1260813600%26key%3Dyt1%26signature%3D158633D23CDC66CC40D171EC8CB48A84B4BCB223.86F67
79A36E14684B367BCCF91588556A871C421%26factor%3D1.25%26id%3Dcf9829e68818de48

how can i match only the second ?

David Masover wrote:

···

On Monday 14 December 2009 04:06:08 am David Villa wrote:

fajljsfjaosfohttp://www.marca.comjafosjodfahttp://www.as.comjfoaasjofja

Any particular context? Or is it actually that random?

i want to extract the diferents url but i try with :
/(http\:\/\/.+com)/ it returns a long match:

   1. http://www.marca.comjafosjodfahttp://www.as.com

If you think about it, that is still a valid URL. You're trying to limit
it
not to URLs, but only to http:// followed by a domain, and then only a
domain
ending in .com -- there are MANY urls that this will break.

If you're OK with that, the basic problem is that . is going to match as
much
as it possibly can (greedy), and it matches any character. The simple
solution
is to make it match as few characters as it can (miserly). You do that
by
putting a question mark after the + or *:

/(http\:\/\/.+?com)/

But again, that's not matching .com, that's matching anything ending in
com.
For example, on this URL:

http://www.broadcom.com/

it will only capture http://www.broadcom. So there's an easy solution --
add
an escaped dot:

/(http\:\/\/.+?\.com)/

That's as much as I want to do with it. I'm guessing what you're trying
to do
is auto-linkify URLs in forum posts, or something like that -- some
problem
that's been solved a million times before, and better, so you should
look for
those solutions. But I won't assume that applies to you...

By the way, if you don't already know:

http://rubular.com/

--
Posted via http://www.ruby-forum.com/\.

8

Yep, but I forgot the %r. Escaping / is ugly :-).
So, thanks for that !

Jesus.

···

On Mon, Dec 14, 2009 at 11:20 AM, Benoit Daloze <eregontp@gmail.com> wrote:

We are quite according in our posts :slight_smile:
This should be the Ruby way, very clear this time !

9

David Villa wrote:

I posted a random string and i understand it better, but the real
question and string is :

But where does this string actually come from? It looks a bit like URLs
but with an extra layer of URL-encoding, in that = appears as %3D, for
example, and some extra numeric prefixes like 18|

So it would be much more helpful to understand what the real structure
of this string is, rather than just guessing, in which case you don't
need to guess about how to decode it.

Removing the first level of escaping:

irb(main):007:0> CGI.unescape(s)

=>
"\"18|http://v14.lscache3.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor&fexp=900034%2C902305&algorithm=throttle-factor&itag=18&ipbits=0&burst=40&sver=3&expire=1260813600&key=yt1&signature=4C33C8CB5787DBA4D111A2E66BFFACCC2E95E0A7.D302F87BF11A4BD47631704E858C762325913921&factor=1.25&id=cf9829e68818de48,34|http://v7.lscache8.c.youtube.com/videoplayback?ip\320.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor&fexp=900034%2C902305&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1260813600&key=yt1&signature=158633D23CDC66CC40D171EC8CB48A84B4BCB223.86F6779A36E14684B367BCCF91588556A871C421&factor=1.25&id=cf9829e68818de48,5|http://v16.lscache8.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%
2Cexpire%2Cip%2Cipbits%2Citag%2Calgrithm%2Cburst%2Cfactor&fexp=900034%2C902305&algorithm=throttle-factor&itag=5&ipbits=0&burst=40&sver=3&expire=1260813600&key=yt1&signature=91FAC179A7C02DD662942CCEB71D8BE05BD7B5D3.BC4BFC97F8BA02519CC817D2541E5D0548BC2C7C&factor=1.25&id=cf9829e68818de48\""

So my total *guess* is that this is a double-quoted string, which
contains comma-separated fields, and each field is of the form nn|URL.
In which case you can unwrap it in stages:

irb(main):011:0> s.sub!(/\A"(.*)"\z/) { $1 }
irb(main):012:0> fields = CGI.unescape(s).split(',')
irb(main):013:0> fields.each { |f| num,url = f.split('|',2); puts
"***",url }; nil

···

***
http://v14.lscache3.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor&fexp=900034%2C902305&algorithm=throttle-factor&itag=18&ipbits=0&burst=40&sver=3&expire=1260813600&key=yt1&signature=4C33C8CB5787DBA4D111A2E66BFFACCC2E95E0A7.D302F87BF11A4BD47631704E858C762325913921&factor=1.25&id=cf9829e68818de48
***
http://v7.lscache8.c.youtube.com/videoplayback?ip�.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgorithm%2Cburst%2Cfactor&fexp=900034%2C902305&algorithm=throttle-factor&itag=34&ipbits=0&burst=40&sver=3&expire=1260813600&key=yt1&signature=158633D23CDC66CC40D171EC8CB48A84B4BCB223.86F6779A36E14684B367BCCF91588556A871C421&factor=1.25&id=cf9829e68818de48
***
http://v16.lscache8.c.youtube.com/videoplayback?ip=0.0.0.0&sparams=id%2Cexpire%2Cip%2Cipbits%2Citag%2Calgrithm%2Cburst%2Cfactor&fexp=900034%2C902305&algorithm=throttle-factor&itag=5&ipbits=0&burst=40&sver=3&expire=1260813600&key=yt1&signature=91FAC179A7C02DD662942CCEB71D8BE05BD7B5D3.BC4BFC97F8BA02519CC817D2541E5D0548BC2C7C&factor=1.25&id=cf9829e68818de48
=> nil

IMO it's far better to use the structure of the input to delimit the
data you're looking for, rather than guessing where the start and end of
each datum is based on what you expect the datum to look like.
--
Posted via http://www.ruby-forum.com/\.

10

Sorry, i forgot this post.

Thanks for all the people.

Finally, i think i end my script to download from youtube and extract
the sound, thanks again.

The script, very simple but very util for me and very interesting to
remember regular expressions which i had forgotten.

www.dvillanueva.com/blog

···

--
Posted via http://www.ruby-forum.com/.