Perl6 grammar rules in ruby

ruby-talk
21

The question I and “anon luker” have isn’t why a library for embedding
grammers wouldn’t be cool, but why do you see a synergy between REGEX
and a grammer?

I haven’t looked at perl6’s grammar, I looked at boost::spirit instead
(a C++ implementation). To me, an embedded grammers/dynamic parsers
looks very useful and fun to use. But, it looks like any good language
(ruby, c++, etc.) should be able to add support as a library.

What is being called a grammar here is very similar to REGEX. I think
the main difference between building up REGEX from other REGEX pieces
and a grammer is that with a REGEX, you pull out pieces of matched text,
and with a grammer, you associate actions to matched text (and sometimes
grammer implementations help you build parse trees, something that is
not possible with a single REGEX). Both grammers and complex REGEX’s are
built up in a BNFy way.

For example, here’s part of the vCard syntax:

# 1*(ALPHA / DIGIT / "=")
NAME    = '[-a-z0-9]+'

# <"> <Any character except CTLs, DQUOTE> <">
QSTR    = '"[^"]*"'
 
# *<Any character except CTLs, DQUOTE, ";", ":", ",">
PTEXT   = '[^";:,]*'
  
# param-value = ptext / quoted-string
PVALUE  = "#{PTEXT}|#{QSTR}"

# param = name "=" param-value *("," param-value)
PARAM = ";(#{NAME})(=?)((?:#{PVALUE})(?:,#{PVALUE})*)"

…

s =~ %r{\s*#{PARAM}\s*}

If that was a grammer, rather than a a REGEX being built up as EBNF, I
would associate actions with the matched text, rather than doing things
with the matched text.

My impression is that in perl, you can inject code to be executed into a
REGEX. Is that true? If so, I would bet you that their grammers are some
kind of wrapper around REGEX exposing essentially a different REGEX
notation, one that is more readable when you have lots of code. They
appear to be two different syntaxes for doing the same thing.

Your example:

str =~ /BEGINEND/

is thoroughly unconvincing. Do this as

str =~ /BEGIN(.*)END/
mygrammer.parse($1)

The classic Lex/Yacc
is built on top of REGEXs, for example. The tokens are specified as a
REGEX, and the grammer as EBNF of tokens.

So, why do you think there needs to be grammers inside a regex?

Cheers,
Sam

Quoteing cc1@cec.wustl.edu, on Fri, Mar 05, 2004 at 10:19:42PM +0900:

···

anon luker wrote:

Could you please explain your take on the relationship between regular
expressions and dynamic parsers? Or regular expressions and grammar
specifications? Your argument that Ruby needs language-level support
for dynamic parsers seems to be based on one of these opaque
relationships.

A major part of all programming language use is parsing text in some
fashion or another, basically it’s a solved problem due to grammars and
regex, it seems logical that if one includes regex support at the
language level, it makes equal sense to give you all the parsing tools
and allow easy interaction between grammar and embedded regex. It makes
it available to the programmer, it makes it easy, and it makes it less
of a creative stretch. IE it allows the programmer to think of certain
programming tasks from a grammar perspective, in the same way one things
of some tasks from a regex perspective. It gives the programmer a
larger vocabulary, without unecessarily complicating things for them.

That is why I am trying to push for this, but I think I’m not quite
following your question. Could you specify a little better, then
perhaps I could answer better :wink:

Charles Comstock