First question: Why does the final statement return 2 instead of nil?
All characters in foo are alphabetic characters in this locale.
Then:
$ echo $LANG
nl_NL
$ cat ./foo
#!/usr/bin/ruby -w
foo = "préférées"
p foo =~ /[^[:alnum:]]/
p foo =~ /\W/
$ ./foo
2
2
Huh?
Second question: Why does the first regex match now return 2 instead of
nil?
To my way of thinking, both statements should always return nil, whether
or not they are typed into irb or run in a stand-alone script. At the
very least, both statements should return the same answer, regardless of
the context.
Maybe there is an initialization in .irbrc that leads to a changed locale inside IRB. Or your IRB belongs to a different Ruby version on that system.
Other than that, I guess you tripped into the wide and wild country of i18n - many strange things can be found there. Maybe \w and \W only treat ASCII [a-z] characters as word characters.
First question: Why does the final statement return 2 instead of nil?
All characters in foo are alphabetic characters in this locale.
Then:
$ echo $LANG
nl_NL
$ cat ./foo
#!/usr/bin/ruby -w
foo = "préférées"
p foo =~ /[^[:alnum:]]/
p foo =~ /\W/
$ ./foo
2
Huh?
Second question: Why does the first regex match now return 2 instead of
nil?
To my way of thinking, both statements should always return nil, whether
or not they are typed into irb or run in a stand-alone script. At the
very least, both statements should return the same answer, regardless of
the context.
Maybe there is an initialization in .irbrc that leads to a changed
locale inside IRB.
Nope; I had hoped it would be that easy, but as you can see from my
snippet of output, I started irb with -f, which bypasses ~/.irbrc.
ENV['LANG'] also prints nl_NL in irb, so that can't be it.
Or your IRB belongs to a different Ruby version on that system.
I compiled it myself, so there has been no mix-and-matching.
Other than that, I guess you tripped into the wide and wild country of
i18n - many strange things can be found there. Maybe \w and \W only
treat ASCII [a-z] characters as word characters.
It does seem that way, as Perl also appears to treat them this way.
However, I'm still puzzled why there's a difference between irb and a
stand-alone script.
Ian
···
On Wed 14 Feb 2007 at 06:45:08 +0900, Robert Klemme wrote:
--
Ian Macdonald | If you are what you eat, I guess that makes
ian@caliban.org | me a cheese danish. -- Anonymous http://www.caliban.org/ |
>
>
My editor is vim and I run it in the nl_NL locale, so it doesn't start
in UTF-8 mode. To double-check:
:set encoding?
encoding=latin1
And if we dump my little script:
$ od -c foo
0000000 # ! / u s r / b i n / r u b y
0000020 - w \n \n f o o = " p r 351 f 351
0000040 r 351 e s " \n p f o o = ~ /
0000060 [ ^ [ : a l n u m : ] ] / \n p
0000100 f o o = ~ / \ W / \n
You can see that it is, indeed, saved as Latin-1, not UTF-8.
The mystery continues.
Ian
···
On Wed 14 Feb 2007 at 08:01:15 +0900, David Balmain wrote:
On 2/14/07, David Balmain <dbalmain.ml@gmail.com> wrote:
>On 2/14/07, Ian Macdonald <ian@caliban.org> wrote:
>> However, I'm still puzzled why there's a difference between irb and a
>> stand-alone script.
>
>Maybe your editor saves the script in UTF-8 format. The irb example
>clearly encodes the string in ISO-8859-1. That could explain the
>difference.
Not nl_NL but en_US.ISO-8859-1. I get the same results as you.
···
On 2/14/07, Ian Macdonald <ian@caliban.org> wrote:
On Wed 14 Feb 2007 at 09:08:17 +0900, David Balmain wrote:
> On 2/14/07, Ian Macdonald <ian@caliban.org> wrote:
> >
> >I should have asked by now, but can anyone else reproduce this with
> >Ruby 1.8.5?
>
> I can reproduce this 1.8.4
Just to be clear, you are confirming that the following code:
foo = "préférées"
p foo =~ /[^[:alnum:]]/
prints nil in irb and 2 in a stand-alone script when in both cases your
locale is preset to nl_NL?
I should have asked by now, but can anyone else reproduce this with
Ruby 1.8.5?
I can reproduce this 1.8.4
Just to be clear, you are confirming that the following code:
foo = "préférées"
p foo =~ /[^[:alnum:]]/
prints nil in irb and 2 in a stand-alone script when in both cases your
locale is preset to nl_NL?
Ian
--
Ian Macdonald | On a clear disk you can seek forever.
ian@caliban.org | http://www.caliban.org/ |
I'm beginning to wonder if the original question is even accurate. Doing nothing more than changing the encoding and re-saving the file (where the value for foo was a cut-n-paste from the email), there doesn't seem to be any discrpeancy between ruby and irb. (This output is from ruby 1.8.5, but 1.8.2 was the same)
As you can see, interactively irb returns nil for that first regex match.
Ian
···
On Wed 14 Feb 2007 at 10:25:10 +0900, Rob Biedenharn wrote:
I'm beginning to wonder if the original question is even accurate.
Doing nothing more than changing the encoding and re-saving the file
(where the value for foo was a cut-n-paste from the email), there
doesn't seem to be any discrpeancy between ruby and irb. (This
output is from ruby 1.8.5, but 1.8.2 was the same)
Phewsh. Combined with the behavior you reported for loading a global
and then matching in IRB, I had feared the world had gone insane. At
least its consistently weird and the regexp match is, in fact, the
culprit.
···
On Feb 14, 8:51 am, Ian Macdonald <i...@caliban.org> wrote:
On Wed 14 Feb 2007 at 23:42:10 +0900, Ian Macdonald wrote:
> On Wed 14 Feb 2007 at 18:00:22 +0900, Robert Klemme wrote:
> > Another idea: maybe the readline lib interferes with encodings somehow
> > in IRB? What happens if you invoke your script from within IRB via "load"?
> It runs as if run from the command line:
> irb(main):001:0> load 'foo'
> 2
> 2
I beg your pardon. I must have had the locale set incorrectly on that
run. It runs as if typed interactively into irb:
Another idea: maybe the readline lib interferes with encodings somehow
in IRB? What happens if you invoke your script from within IRB via "load"?
It runs as if run from the command line:
irb(main):001:0> load 'foo'
2
I beg your pardon. I must have had the locale set incorrectly on that
run. It runs as if typed interactively into irb:
$ irb
irb(main):001:0> load 'foo'
nil
2
Phewsh. Combined with the behavior you reported for loading a global
and then matching in IRB, I had feared the world had gone insane. At
least its consistently weird and the regexp match is, in fact, the
culprit.
Why don't you just find out which characters are in the [:alnum:] and \w sets?
