Regular expressions question

ruby-talk
21

Just to add another voice to the maelstrom: I have never coded Perl
and don't particular like it, but I almost always use the $1 variables
in Ruby. I have been coding Ruby for over four years and I rarely find
instances where MatchData is all that much better than =~ and the $1
variables.

They are just more convenient in the typical case:

str="abc"
re=/(.)(.)(.)/

if re =~ str
  p $2
end

# Versus:

if (md = re.match(str))
  p md[2]
end

Ryan

···

On 12/14/05, Jeff Wood <jeff.darklight@gmail.com> wrote:

You should be able to tell who this message is meant for:

PLEASE stop sending out code that uses any of the perl ${x} variables ...

They are ugly and have no place in Ruby ... they are only provided to
make the transition of Perl people easier ...

22

Ross Bamford wrote:

···

On Thu, 15 Dec 2005 00:16:52 -0000, Jeff Wood <jeff.darklight@gmail.com> > wrote:

> You should be able to tell who this message is meant for:
>

Why not just address me directly?

> PLEASE stop sending out code that uses any of the perl ${x} variables ...
>

Well, okay. No need to shout though, is there?

Just trying to put a bit back, you know?

Ross, don't pay too much attention to unreasonable fanatics.

The first edition of the Pickaxe says:

"Having said all this, we have to 'fess up. Andy and Dave normally
use the $-variables rather than worrying about MatchData objects.
For everyday use, they just end up being more convenient.
Sometimes we just can't help being pragmatic."

23

James Edward Gray II wrote:

I see a lot of code running the Ruby Quiz and I feel quite confident
saying that the Regexp variables are far more common than MatchData. I
don't think that says anything bad about the latter, but it does tell
me that you are in the minority. :wink:

If the majority here uses globally scoped variables to store locally
used values, then that doesn't say anything good for this portion of
the Ruby community.

···

--
Neil Stevens - neil@hakubi.us

'A republic, if you can keep it.' -- Benjamin Franklin

24

Ross Bamford wrote:

···

On Thu, 15 Dec 2005 00:16:52 -0000, Jeff Wood > <jeff.darklight@gmail.com> wrote:

You should be able to tell who this message is meant for:

Why not just address me directly?

PLEASE stop sending out code that uses any of the perl ${x} variables ...

Well, okay. No need to shout though, is there?

Just trying to put a bit back, you know?

Because it wasn't *just* directed at you ... there have been other posts that included those same "warts".

And I didn't mean to shout, I just meant to exaggerate the please .... *Please* would have been more appropriate.

j.

25

James Edward Gray II wrote:

You should be able to tell who this message is meant for:

Yes, I recognize that you are probably speaking at least in part to me, since I did that in this very thread. You can call me by name if you like. I'm a big boy and I can take it. :wink:

PLEASE stop sending out code that uses any of the perl ${x} variables ...

Hang on there Mr. Code Police. Let's not lay down the law down too heavily before we get into this...

They are ugly and have no place in Ruby ... they are only provided to
make the transition of Perl people easier ...

I seriously doubt those variables were invented in Perl. They are a common feature to many Regular Expression implementation and I'm not sure they are even that ugly. $1 holds what was grabbed by the first set of parenthesis. Fairly logical.

Please teach people to use MatchData objects ...

I also showed a MatchData example.

I've used them a time or two, but honestly, they just don't feel right to me. I've stopped using the default variable, I'm using a two-space tab, etc. I'm Ruby assimilated, but I just like the Regexp- linked variables.

I see a lot of code running the Ruby Quiz and I feel quite confident saying that the Regexp variables are far more common than MatchData. I don't think that says anything bad about the latter, but it does tell me that you are in the minority. :wink:

We won't yell at you for using MatchData, if you'll provide the same consideration...

James Edward Gray II

Quite simply, Ruby is *supposed* to be about consistency ... Having the "everything is an object, principal of least surprise" mantra, then using these which act like a global ( $ ) but aren't actually ( local scope ) is just vile.

That's why I have a problem with them. If the community uses them, well, that's their option, I'm just one that's all for consistency, always, as much as possible... It tends to make things more generic and able to handle change better.

I can't speak as to whether Perl was the first language to do the ${x} variables ... but, it ( so far of the languages I've learned ) uses it heavily, and it contributes to all of the punctuation soup that we all left Perl to get away from... ( again I'm speaking generally, but I could also again be wrong ... that's one of the uglies I left because of ).

... Not trying to be language police, I just really love the MatchData, and find it MUCH easier to deal with. Then you can keep your datasets from multiple matches around ... to me it *is* easier to read ... instead of ... $1 ... where'd that come from ... I didn't assign a glob... oh that's right ...

Anyways, I'm sorry to have causes this thread to go on this long ... I just really thought more of the people on the list would step up and say, yeah, those are some very ugly warts and we don't use them ... but apparently, I was wrong.

I'll shut up now.

j.

···

On Dec 14, 2005, at 6:16 PM, Jeff Wood wrote:

26

You should be able to tell who this message is meant for:

Yes, I recognize that you are probably speaking at least in part to me, since I did that in this very thread. You can call me by name if you like. I'm a big boy and I can take it. :wink:

(Sorry, James, I assumed that was directed at me alone)

PLEASE stop sending out code that uses any of the perl ${x} variables ...

[...]

Please teach people to use MatchData objects ...

I also showed a MatchData example.

I think the $1, $2 stuff just hit Jeff's anger button, because I had MatchData down at the bottom of my examples too but it seemed to be missed.

I've used them a time or two, but honestly, they just don't feel right to me. I've stopped using the default variable, I'm using a two-space tab, etc. I'm Ruby assimilated, but I just like the Regexp-linked variables.

Same here - I like to stick to the convention of whatever language I'm using. To begin with I did use a lot more Perl-like stuff (I started with Ruby using an old Perl book. Don't ask...) but now i'm starting to find the Rubyisms that work for me ($LOAD_PATH instead of $: and so on).

It seems sensible to me to save the MatchData stuff for the times when it's needed. Java's regexp support is purely OOP-based (of course, being Java ;)) and I can't remember when I last used it for something simple - it's just nowhere near as convenient as matching a literal regexp and using the numbered variables...

Cheers,
Ross

···

On Thu, 15 Dec 2005 03:35:15 -0000, James Edward Gray II <james@grayproductions.net> wrote:

On Dec 14, 2005, at 6:16 PM, Jeff Wood wrote:

--
Ross Bamford - rosco@roscopeco.remove.co.uk
"\e[1;31mL"

27

ako... wrote:

thank you. yes, it seems to be the only way. just that it is a shame
that we have to match the same expression again! the information was
available already, it was just discarded during the first match in
your sample.

I still didn't get what exactly you want. Does this help?

'a,b ,c'.split /\s*,\s*/

=> ["a", "b", "c"]

Kind regards

    robert

28

MatchData can be quite convenient too

str = "foobar 2005-12-15"
if md = /(\S+) (\d+)-(\d+)-(\d+)/.match(str)
  name, year, month, day = md.captures # => ["foobar", "2005", "12", "15"]
  name # => "foobar"
  # ....
end

If you want to name the captures,
    md.captures
looks better than
    $1, $2, $3, $4

···

On Fri, Dec 16, 2005 at 05:05:00AM +0900, Ryan Leavengood wrote:

Just to add another voice to the maelstrom: I have never coded Perl
and don't particular like it, but I almost always use the $1 variables
in Ruby. I have been coding Ruby for over four years and I rarely find
instances where MatchData is all that much better than =~ and the $1
variables.

They are just more convenient in the typical case:

--
Mauricio Fernandez

29

William James wrote:

Ross, don't pay too much attention to unreasonable fanatics.

The first edition of the Pickaxe says:

"Having said all this, we have to 'fess up. Andy and Dave normally
use the $-variables rather than worrying about MatchData objects.
For everyday use, they just end up being more convenient.
Sometimes we just can't help being pragmatic."

How convenient that you quote that without quoting the drawbacks listed
first...

Sheesh, if you want perl, use perl.

···

--
Neil Stevens - neil@hakubi.us

'A republic, if you can keep it.' -- Benjamin Franklin

30

I really think you are blowing the issue out of proportion. A Regexp is generally checked in one-line and the variables used in the next. It doesn't make a lot of sense to hold on to them for twenty lines after you make the check.

Also, I believe they are thread-local variables are they not? (I'm honestly asking.) If so, I don't see a lot of concern about them being stomped on before they are used.

James Edward Gray II

···

On Dec 14, 2005, at 9:42 PM, Neil Stevens wrote:

James Edward Gray II wrote:

I see a lot of code running the Ruby Quiz and I feel quite confident
saying that the Regexp variables are far more common than MatchData. I
don't think that says anything bad about the latter, but it does tell
me that you are in the minority. :wink:

If the majority here uses globally scoped variables to store locally
used values, then that doesn't say anything good for this portion of
the Ruby community.

31

:slight_smile: Thanks. I don't feel nearly so bad about being too lazy to get a MatchData now :wink:

···

On Thu, 15 Dec 2005 01:23:14 -0000, William James <w_a_x_man@yahoo.com> wrote:

Ross Bamford wrote:

On Thu, 15 Dec 2005 00:16:52 -0000, Jeff Wood <jeff.darklight@gmail.com> >> wrote:

> PLEASE stop sending out code that uses any of the perl ${x} variables ...
>

Ross, don't pay too much attention to unreasonable fanatics.

The first edition of the Pickaxe says:

"Having said all this, we have to 'fess up. Andy and Dave normally
use the $-variables rather than worrying about MatchData objects.
For everyday use, they just end up being more convenient.
Sometimes we just can't help being pragmatic."

--
Ross Bamford - rosco@roscopeco.remove.co.uk
"\e[1;31mL"

32

Ross Bamford wrote:

You should be able to tell who this message is meant for:

Why not just address me directly?

PLEASE stop sending out code that uses any of the perl ${x} variables
...

Well, okay. No need to shout though, is there?

Just trying to put a bit back, you know?

Because it wasn't *just* directed at you ... there have been other posts
that included those same "warts".

Ok, sorry. I'm kind of paranoid :wink:

And I didn't mean to shout, I just meant to exaggerate the please ....
*Please* would have been more appropriate.

I wondered that after I posted. Just my old Fidonet reflexes kicking in I suppose :slight_smile:

Just to say, though, that I tend to feel when someone asks a question, that I should give them all the alternative answers I have, and let them choose what suits them. If we really don't want people to use something (which I don't agree with in this case btw) then the way to do that is to make sure they completely understand that thing - only then can they choose for themselves whether it's good or not.

(I'm thinking specifically about the anti-pattern wars that have consumed a lot of good work in Java over the past few years).

So anyway, that's what I did. It's just natural I think to start with the shortest way ($1), move to the longer ways ($~[1], $~.captures[0]) and finally the 'long' way (/.../.match etc). I did slip in a subtle hint at the end that MatchData was worth looking up, with the $~.inspect line...

Cheers,
Ross

···

On Thu, 15 Dec 2005 04:10:17 -0000, Jeff Wood <jeff.darklight@gmail.com> wrote:

On Thu, 15 Dec 2005 00:16:52 -0000, Jeff Wood >> <jeff.darklight@gmail.com> wrote:

--
Ross Bamford - rosco@roscopeco.remove.co.uk
"\e[1;31mL"

33

Quoting Neil Stevens <neil@hakubi.us>:

If the majority here uses globally scoped variables to store
locally used values, then that doesn't say anything good for
this portion of the Ruby community.

Were you aware that $1 and friends aren't actually globally scoped?

-mental

34

Now that I've read the responses in this thread a few times, I think
I understand what he wants to do. And I don't think it can be done
via scan.

First: He wants a single regex which will verify the syntax of an
entire line. So, first he wants a true/false value, saying "The line
is valid, or it is not valid". Never mind any values in the line, just
"is the line *completely valid*?".

Then, if the line is valid, he wants to break out individual pieces
of what was scanned, and he wants to do that without re-doing
any of the scans he did in the first regex. The trick is that some
of those pieces are a repeating group, such as /(\s\w)*/.

What is confusing us is that he describes this using a simple
example, and when we solve the simple example he then says
"you don't get the bigger picture!". Ugh.

Let me give an example, and see if someone can solve it. My
example might still be something other than what he's thinking
of, but maybe it will help.

Let's say I'm expecting command lines of the form:
   first word is either 'copy' or 'duplicate'
   followed by one or more words
   followed by the word 'before' or 'after'
   followed by one or more words

So I could do the first step with the regexp:

  /^(copy|duplicate) \s+ (\w+\s+)+ (before|after) \s+ (\w+\s*)+ $/x

(hopefully I've done that right!). *IF* that matches, then I know
the entire line is valid. Then, after I know the line is valid, I want
the array of source-words, and the array of destination-words
which were matched. I want to do that by picking out information
in Matchdata, not by doing a new scan. The thing is, I don't think
I have a way of knowing how many times the first '(\d+\s+)+' was
matched. So I can't just do a slice of $~.captures because I don't
know what the starting and ending indexes of that slice would be.
I could put another set of parenthesis around the two repeating
groups:

  /^(copy|duplicate) \s+ ((\w+\s+)+) (before|after) \s+ ((\w+\s*)+) $/x

But that doesn't really give me two separate arrays of the
individual values that made up each group. It just matches
each group as a whole.

Given two data lines of:
    copy apple pear plum peach after bill bob
    duplicate tomato before joe alice alfred tommy jane

in the first case I want a way to set two arrays:
    srcfood = ["apple ", "pear ", "plum ", "peach "]
    destword = ["bill ", "bob"]
from the first line, and
    srcfood = ["tomato "]
    destword = ["joe ", "alice", "alfred ", "tommy ", "jane"]
from the second line.

I'll agree this is a weird example, but I think it shows the issue.
If I apply the above pattern to the first line, I'll see a Matchdata
result where:

$~.captures ==
   ["copy", "apple pear plum peach ", "peach ", "after", "bill bob", "bob"]

Notice: There isn't *any* element which contains a value of just "apple ",
or just "pear ", or just "plum ", even though the regex obviously had to
match each one of those.

···

On 12/15/05, Robert Klemme <bob.news@gmx.net> wrote:

ako... wrote:
> thank you. yes, it seems to be the only way. just that it is a shame
> that we have to match the same expression again! the information was
> available already, it was just discarded during the first match in
> your sample.

I still didn't get what exactly you want. Does this help?

>> 'a,b ,c'.split /\s*,\s*/
=> ["a", "b", "c"]

--
Garance Alistair Drosehn = drosihn@gmail.com
Senior Systems Programmer or gad@FreeBSD.org
Rensselaer Polytechnic Institute; Troy, NY; USA

35

Jeff Wood wrote:

I'll shut up now.

I think that's exactly what some people want you to do. People don't
want to be told that they're lousy coders, and their poor practices have
only been made to work through a special case in the language interpreter.

···

--
Neil Stevens - neil@hakubi.us

'A republic, if you can keep it.' -- Benjamin Franklin

36

Jeff Wood wrote:

Quite simply, Ruby is *supposed* to be about consistency ... Having the "everything is an object, principal of least surprise" mantra, then using these which act like a global ( $ ) but aren't actually ( local scope ) is just vile.

You have made a common mistake, you are thinking Ruby is meant for your principle of least surprise, when it is matz's least surprise...

Quoting Matz,

"Besides that, he doesn't understand what POLS means. Since someone
will surprise for any arbitrary choice, it is impossible to satisfy
"least surprise" in his sense. The truth is two folds: a) when there
are two or more choices in the language design decision, I take the
one that makes _me_ surprise least. "

http://groups.google.com/group/comp.lang.ruby/msg/875965f29cfb77bc

Zach

37

James Edward Gray II wrote:

Also, I believe they are thread-local variables are they not? (I'm
honestly asking.) If so, I don't see a lot of concern about them being
stomped on before they are used.

Actually, I just looked it up, and according to "Programming Ruby,"
$1-$9 are "local to the current scope."

My mistake, heh. I wonder how many who use them know that, though, and
how many just do it without checking because it's popular in perl or
popular on here.

···

--
Neil Stevens - neil@hakubi.us

'A republic, if you can keep it.' -- Benjamin Franklin

38

Quoting James Edward Gray II <james@grayproductions.net>:

Also, I believe they are thread-local variables are they not?
(I'm honestly asking.) If so, I don't see a lot of concern
about them being stomped on before they are used.

They're method-local.

def foo
  "abc" =~ /(a)/
  p $1
end

foo => "a"
p $1 => nil

-mental

39

Neil Stevens <neil@hakubi.us> writes:

William James wrote:

Ross, don't pay too much attention to unreasonable fanatics.

The first edition of the Pickaxe says:

"Having said all this, we have to 'fess up. Andy and Dave normally
use the $-variables rather than worrying about MatchData objects.
For everyday use, they just end up being more convenient.
Sometimes we just can't help being pragmatic."

How convenient that you quote that without quoting the drawbacks listed
first...

Which drawbacks?

···

Neil Stevens - neil@hakubi.us

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

40

Garance A Drosehn wrote:

First: He wants a single regex which will verify the syntax of an
entire line. So, first he wants a true/false value, saying "The line
is valid, or it is not valid". Never mind any values in the line, just
"is the line *completely valid*?".

Then, if the line is valid, he wants to break out individual pieces
of what was scanned, and he wants to do that without re-doing
any of the scans he did in the first regex. The trick is that some
of those pieces are a repeating group, such as /(\s\w)*/.

What is confusing us is that he describes this using a simple
example, and when we solve the simple example he then says
"you don't get the bigger picture!". Ugh.

Let me give an example, and see if someone can solve it. My
example might still be something other than what he's thinking
of, but maybe it will help.

Let's say I'm expecting command lines of the form:
   first word is either 'copy' or 'duplicate'
   followed by one or more words
   followed by the word 'before' or 'after'
   followed by one or more words

So I could do the first step with the regexp:

  /^(copy|duplicate) \s+ (\w+\s+)+ (before|after) \s+ (\w+\s*)+ $/x

(hopefully I've done that right!). *IF* that matches, then I know
the entire line is valid. Then, after I know the line is valid, I want
the array of source-words, and the array of destination-words
which were matched. I want to do that by picking out information
in Matchdata, not by doing a new scan. The thing is, I don't think
I have a way of knowing how many times the first '(\d+\s+)+' was
matched. So I can't just do a slice of $~.captures because I don't
know what the starting and ending indexes of that slice would be.
I could put another set of parenthesis around the two repeating
groups:

  /^(copy|duplicate) \s+ ((\w+\s+)+) (before|after) \s+ ((\w+\s*)+) $/x

But that doesn't really give me two separate arrays of the
individual values that made up each group. It just matches
each group as a whole.

Given two data lines of:
    copy apple pear plum peach after bill bob
    duplicate tomato before joe alice alfred tommy jane

in the first case I want a way to set two arrays:
    srcfood = ["apple ", "pear ", "plum ", "peach "]
    destword = ["bill ", "bob"]
from the first line, and
    srcfood = ["tomato "]
    destword = ["joe ", "alice", "alfred ", "tommy ", "jane"]
from the second line.

I'll agree this is a weird example, but I think it shows the issue.
If I apply the above pattern to the first line, I'll see a Matchdata
result where:

$~.captures ==
   ["copy", "apple pear plum peach ", "peach ", "after", "bill bob", "bob"]

DATA.each {|line| line.chomp!
  md =
    /^(?:copy|duplicate) \s+
      ((?:\w+\s+)+)
      (?:after|before) \s+
      ((?:\w+\s*)+) $
    /x.match( line )
  p md.captures
  src_food = md.captures.first.split
  dest_word = md.captures.last.split
  p src_food, dest_word
}

__END__
copy apple pear plum peach after bill bob
duplicate tomato before joe alice alfred tommy jane

----- output: -----

["apple pear plum peach ", "bill bob"]
["apple", "pear", "plum", "peach"]
["bill", "bob"]
["tomato ", "joe alice alfred tommy jane"]
["tomato"]
["joe", "alice", "alfred", "tommy", "jane"]