Which Regex-Engine will be used in Ruby 1.8.3 Release?

One short question.

Which Regular Expression Engine will be used in the Ruby 1.8.3 release (Is it still the old one or Onigurama
2.4.*)?

Onigurama would be fine - I don't talk here about the exotic usages of recursive calls - simply the look
behind patterns are very useful an not implemented in the old engine.

···

--
Wolfgang Nádasi-Donner
wonado@donnerweb.de

Hi,

···

In message "Re: Which Regex-Engine will be used in Ruby 1.8.3 Release?" on Fri, 29 Jul 2005 20:16:01 +0900, "Wolfgang N疆asi-Donner" <wonado@donnerweb.de> writes:

One short question.

Which Regular Expression Engine will be used in the Ruby 1.8.3 release (Is it still the old one or Onigurama
2.4.*)?

One short answer: old one.

              matz.

Yukihiro Matsumoto <matz@ruby-lang.org> writes:

Hi,

>One short question.
>
>Which Regular Expression Engine will be used in the Ruby 1.8.3 release (Is it still the old one or Onigurama
>2.4.*)?

One short answer: old one.

Will there be a 1.8.x release with Oniguruma being the default engine?

···

In message "Re: Which Regex-Engine will be used in Ruby 1.8.3 Release?" > on Fri, 29 Jul 2005 20:16:01 +0900, "Wolfgang N疆asi-Donner" <wonado@donnerweb.de> writes:

matz.

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

Wolfgang Nadasi-Donner wrote:

In addition I have to excuse me for using always the wrong word 'Onigurama'. Meanwhile I recognized, that the
correct name is 'Oniguruma' (unfortunately my knowlede about the japanese language is somehow 10e-20% - I am
even a bad Go-player).

An understandable and common error.

Just remember you can't spell "Oniguruma" without "guru." :wink:

Cheers,
Hal

Hi,

···

In message "Re: Which Regex-Engine will be used in Ruby 1.8.3 Release?" on Sat, 30 Jul 2005 19:09:53 +0900, Christian Neukirchen <chneukirchen@gmail.com> writes:

Will there be a 1.8.x release with Oniguruma being the default engine?

It's not planned, for compatibility's sake. Any opinion?

              matz.

Hi all,

Yukihiro Matsumoto wrote:

···

In message "Re: Which Regex-Engine will be used in Ruby 1.8.3 Release?" > on Sat, 30 Jul 2005 19:09:53 +0900, Christian Neukirchen <chneukirchen@gmail.com> writes:

>
> >Will there be a 1.8.x release with Oniguruma being the default engine?
>
> It's not planned, for compatibility's sake. Any opinion?
>
> matz.

That (partly) depends on when Oniguruma will become the default regex engine otherwise, I'd guess.

Happy rubying

Stephan

Yukihiro Matsumoto <matz@ruby-lang...> wrote:

Christian Neukirchen writes:
> >Will there be a 1.8.x release with Oniguruma being the default engine?

It's not planned, for compatibility's sake. Any opinion?

Compatibility in the sense that an invalid Regexp for the current engine
could be interpreted as something exotic by Oniguruma ?

Would something like the following be possible? :

1.8.3 (say, from Dec 2005) include _both_ (i.e. selectable)

  Regexp::service_level(0) (DEFAULT) - Current engine
  Regexp::service_level(1) - Oniguruma ( strict emulation of level(0) )**
  Regexp::service_level(2) - Oniguruma ( full )

.... where service_level(n) calls Init_Regexp_n()
(The transitional method name would *not* become part of the language.)

1.8.4 (or after 6 / 12 months)

  Regexp::service_level(0) - Current engine ( in case of emergency :wink:
  Regexp::service_level(1) (DEFAULT) - Oniguruma ( strict emulation of level(0) )**
  Regexp::service_level(2) - Oniguruma ( full )

1.8.5 (?)

  Regexp::service_level(1) - Oniguruma ( strict emulation of level(0) )**
  Regexp::service_level(2) (DEFAULT) - Oniguruma ( full )

A big advantage would be to enable behavioural comparisons
from a script without switching between 1.8 / 1.9 ?

Quite conservative, too. No-one needs to be concerned that it's there.

** If I read correctly, the Oniguruma team have allowed for
   building versions tailored to a variety of users.

   Actually, I'm still going to post but the difference between
   level(1) & (2) would require adding a lot of almost duplicate
   code to the Ruby distribution _or_ an awful lot of added logic
   which I wouldn't wish on anyone.

   Some clever folks out there though :slight_smile:

daz

I guess I can see why that's the case, but I'm very excited about the new engine and can't wait for it to make an official appearance.

I know Oniguruma supports more features, but does it not support all of the current engine?

James Edward Gray II

···

On Jul 30, 2005, at 6:20 AM, Yukihiro Matsumoto wrote:

>Will there be a 1.8.x release with Oniguruma being the default engine?

It's not planned, for compatibility's sake. Any opinion?

Hi,

···

In message "Re: Which Regex-Engine will be used in Ruby 1.8.3 Release?" on Sat, 6 Aug 2005 06:54:08 +0900, James Edward Gray II <james@grayproductions.net> writes:

It's not planned, for compatibility's sake. Any opinion?

I guess I can see why that's the case, but I'm very excited about the
new engine and can't wait for it to make an official appearance.

I know Oniguruma supports more features, but does it not support all
of the current engine?

Two reasons:

  * no one have verified regular expression compatibility in detail before.
  * no one have proven the stability of the new engine.

It would not be merged unless someone pay the cost to verify the new
engine.

              matz.

snip >>>>>

"Yukihiro Matsumoto" <matz@ruby-lang.org> schrieb im Newsbeitrag
news:1123316581.425077.14779.nullmailer@x31.priv.netlab.jp...
....

In message "Re: Which Regex-Engine will be used in Ruby 1.8.3 Release?"

....

Two reasons:

  * no one have verified regular expression compatibility in detail before.
  * no one have proven the stability of the new engine.

It would not be merged unless someone pay the cost to verify the new
engine.

snap >>>>>

I am interested in a early integration of Oniguruma into Ruby too. I will be in a Health cure clinic for
cancer follow-up treatment starting end of next week for up to 6 weeks, accompanied by my good old Notebook,
having a lot of time, but without regular access to the internet.

The installation for making tests is extracted from 'ruby-1.9.0-20050722-i386-mswin32.zip', all additionally
required dlls are installed too, and it works on Windows2000.

I can make the following tests:

- Test of all regular expression patterns listed in the Pickaxe book and several combinations of them

- Test of all regular expression options listed in the Pickaxe Book and several combinations of them

- Test of everything related to the classes Regex and MatchData, and relevant parts of class String

- Test of other regular expression usages (e.g. split)

- Test related to features of Oniguruma, which are not part of the old pattern matching engine

For tests related to internationalization I can take care of Unicode UTF-8 (hopefully).

If there is any interest I can zip together all sources, comments and console outputs and find a way to
transfer the data somehow via internet to ??? (if I don't find an internet cafe, it is possible to send the
data on diskette via snail-mail to someone who can do the job).

This can be done up to end of september / beginning of october 2005 (year should be named in IT ;-)) ).

To whom should the data be sent? (Please answer before 10. of october in comp.lang.ruby or directly to:
'wonado AT donnerweb DOT de')

Wolfgang

Wolfgang Nádasi-Donner ha scritto:
<snip>

I am interested in a early integration of Oniguruma into Ruby too. I will be in a Health cure clinic for
cancer follow-up treatment starting end of next week for up to 6 weeks, accompanied by my good old Notebook,
having a lot of time, but without regular access to the internet.

I hope you'll be home safely soon

The installation for making tests is extracted from 'ruby-1.9.0-20050722-i386-mswin32.zip', all additionally
required dlls are installed too, and it works on Windows2000.

I can make the following tests:

- Test of all regular expression patterns listed in the Pickaxe book and several combinations of them

- Test of all regular expression options listed in the Pickaxe Book and several combinations of them

- Test of everything related to the classes Regex and MatchData, and relevant parts of class String

- Test of other regular expression usages (e.g. split)

- Test related to features of Oniguruma, which are not part of the old pattern matching engine

notice that there are some more place you can consider:
- rubicon[1] used to be a test suite for the interpreter, ATM it is somewhat out of sync, but most of the regex tests still pass or have minor problems (i.e. a warning message is changed).
Sadly, cvs-HEAD seem to fail different tests than ruby_1_8, so it may be hard to replace the current engine ad it is now.

It would be nice if rubicon could be updated in the process :slight_smile:

the second place is
- the test suite for the pure ruby regexp package[1] from
Simon Strandgaard. I think many tests may be uninteresting or duplicated WRT rubicon, but maybe it could be useful

About reporting: maybe you can use the bug tracker for ruby at rubyforge?

[1]http://rubyforge.org/projects/rubytests/
[2]http://rubyforge.org/projects/aeditor/