Syntax checker wtf?

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

Cheers,
Newb

···

--
Posted via http://www.ruby-forum.com/.

No, it's just that Ruby is so flexible that your incorrect syntax
could well be right if you tidied it up at the end.

···

On 8/18/06, Firstname Surname <rubyforum@wendlink.com> wrote:

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

--
Phillip Hutchings
http://www.sitharus.com/

Firstname Surname wrote:

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens with missing chars, extra ., all sorts of minor syntax errors, not just missing 'end's. I found myself copy and pasting functions, checking, then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at 1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line # a
high % of the time?

I *think* this is at least partly due to Ruby's convenient and very flexible syntax. IOW, the parser has no means to detect the error line as it could be in several places - likely too many to report. Also, this is how parsers work. From my limited insight into the parser generation business it would require a) a different parser generator that is much smarter or b) a lot of effort that it's not worth IMHO.

I'll have to add that this is rarely an issue for me. Maybe you should use an editor with automatic bracket closing and "end" insertion.

Kind regards

  robert

"Firstname Surname" <rubyforum@wendlink.com> wrote in message
news:80ebfecc5915f6feffed6fa20fde07e8@example.com...

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens
with missing chars, extra ., all sorts of minor syntax errors, not just
missing 'end's. I found myself copy and pasting functions, checking,
then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at
1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

    While I'm a newbie myself, I'm willing to guess that there's little you
can do about this problem except to try to relax and be a little more
careful with your code.
    To be honest, I really don't think this problem is particularly bad in
Ruby. I don't know exactly what kind of errors you're making (and perhaps
you can elaborate some more on that) but even the latest Microsoft C++
compiler often gives less than helpful clues on the source of the problem,
or even it's location! Brace mismatching and quote mismatching are two
errors that neither parsers can nail down to your satisfaction and I get the
sense that these constitute the majority of your problems (again, please
elaborate on this). The former can be alleviated by proper indentation and
the latter can be totally eliminated by syntax highlighting, although I
don't use this technique myself (since I'm a lazy gvim user).

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker

...

1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

I get messages like this occasionally. I've found that a good editor is the most helpful thing in finding these errors. I use TextMate on my Mac. If I get an error like you metioned, I just give it the command to reindent everything, then look for the "funny indent". You can also do the same in Emacs with ruby-mode. I'm sure jEdit, Eclipse and dozens of other editors have similar features. Auto-indenting of lines as you enter them is helpful too. (Use ^J instead of [return] in Emacs)

Learning a new language and framework can be frustrating at times, but I encourage you not to be too hasty in giving up and moving on.

Regards,
   JJ

···

On Fri, 18 Aug 2006 02:10:48 -0400, Firstname Surname <rubyforum@wendlink.com> wrote:

--
Using Opera's revolutionary e-mail client: http://www.opera.com/mail/

Ever wonder why so many languages have explicit statement termination characters (frequently ";")? This is one of the
main reasons. I wrote a scripting language myself and, in the process, realized I could write a perfectly good
grammar without those statement termination characters. As we started using it though, we started encountering many
situations like you describe. When something is left out, the parser just keeps looking, and looking, and looking, trying to
make sense of the input stream, and eventually runs out of input and has to report an error at the current location (EOF).
However, as soon as a added a ";" to the grammar as a statement termination, the parser had better places to stop trying to
make sense of the input stream. (If it encountered a ";" and did not have a valid statement, it could report the error, then and
there, reset it's state and continue parsing.)

So, IMHO, the primary cause is the absence of those pesky statement terminators.

Firstname Surname wrote:

···

I'm new to Ruby and RoR; I was messing around with it today and had
trouble with error codes. Specifically, the ruby syntax checker
frequently spits out 'syntax error' with the last line of the code as
the error line (the error is nowhere near the last line). This happens with missing chars, extra ., all sorts of minor syntax errors, not just missing 'end's. I found myself copy and pasting functions, checking, then pasting again to find the error, which is pretty ridiculous.

This is something I'd expect in a pre 1.0 compiler, but ruby is at 1.8.4; is the
ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line # a
high % of the time?

Cheers,
Newb

Phillip Hutchings wrote:

···

On 8/18/06, Firstname Surname <rubyforum@wendlink.com> wrote:

ruby syntax checker just really primitive?
This isn't very user friendly, especially for a newbie, is there a more
precise syntax checker that can actually point closer to the error line
# a
high % of the time?

No, it's just that Ruby is so flexible that your incorrect syntax
could well be right if you tidied it up at the end.

It's not a bug, it's a feature! Geeze, django is looking better already.

--
Posted via http://www.ruby-forum.com/.

As "Firstname Surname" said, the error messages from the ruby interpreter are not really significant, they could be more verbose. Maybe Ruby 2.0 will do it better.
However, a good editor may help, but it can not be the solution, or?

Regards, Gregor

Robert Klemme schrieb:

···

I'll have to add that this is rarely an issue for me. Maybe you should use an editor with automatic bracket closing and "end" insertion.

Kind regards

    robert

Robert Klemme <shortcutter@googlemail.com> writes:

I *think* this is at least partly due to Ruby's convenient and very
flexible syntax. IOW, the parser has no means to detect the error
line as it could be in several places - likely too many to report.
Also, this is how parsers work. From my limited insight into the
parser generation business it would require a) a different parser
generator that is much smarter or b) a lot of effort that it's not
worth IMHO.

I'll note that perl has a similarly flexible syntax, yet I don't hear
people new to perl complaining about this. Perhaps this is because
perl has more to complain about, from a newbie perspective, but I
think it's also because perl gives better syntax error messages. For
example, if you get a runaway unclosed string or regexp (//) operator,
the perl interpreter will, in the syntax error, also say something
like "possible runaway string beginning line NN".

This seems to me like it would be a simple addition to the ruby
parser, to mention the line number of the thing it's trying to close
when expecting tEND, or ']', or some other nested thing that can go
off the end of the file.

Mike Cargal wrote:

So, IMHO, the primary cause is the absence of those pesky statement terminators.

Ruby has a statement terminator, it's just optional. If this is an issue for you, feel free to use it.

Feeding the file:

1 class Foo
2
3 def baz
4 puts 'quux';
5 end
6
7 def foo
8 puts 'bar';
9 return Integer("10";
10 end
11
12 def fred
13 puts 'barney'
14 end
15
16 end

to ruby results in:
  
   foo.rb:9: syntax error
   foo.rb:16: syntax error

And the parser fails rather cleanly on the first unclosed paren. It's not quite as if the syntax were strict, and I'm sure some things can slip through, but it's a help.

Ruby isn't about enforcing rules and making decisions for you, it's about TIMTOWDI and adopting conventions that help you personally from the options available.

David Vallner

Gregor Kopp wrote:

As "Firstname Surname" said, the error messages from the ruby
interpreter are not really significant, they could be more verbose.
Maybe Ruby 2.0 will do it better.
However, a good editor may help, but it can not be the solution, or?

Regards, Gregor

Robert Klemme schrieb:

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

The trouble I had was with a line that had an extra '.' like:
@entry = Entry.find(params[:id].
instead of
@entry = Entry.find(params[:id]

and another error that i think was just a missing bracket maybe like
@entry = Entry.find(params[:id

I've used many different languages with various levels of syntax error
help, but never one this bad. I just wrote a quick script to remove
functions from 'def' to 'end' in order and check the syntax over, thus
quickly finding which function the error is in. This is a lot better
than looking through a couple hundred line controller file. If I can do
this in 10 minutes, I can't help but think it's just laziness on the
part of the syntax checker not to provide at least hints where it thinks
the problem might be.
I understand that ruby is flexible (I'm not not sure this is a merit
when using for a large web app), but the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

···

--
Posted via http://www.ruby-forum.com/.

Daniel Martin wrote:

Robert Klemme <shortcutter@googlemail.com> writes:

I *think* this is at least partly due to Ruby's convenient and very
flexible syntax. IOW, the parser has no means to detect the error
line as it could be in several places - likely too many to report.
Also, this is how parsers work. From my limited insight into the
parser generation business it would require a) a different parser
generator that is much smarter or b) a lot of effort that it's not
worth IMHO.

I'll note that perl has a similarly flexible syntax,

I don't think so. For example, you must terminate statements with a ";" (or did that change in the latest version?) and you cannot omit parentheses. That makes up for a significant difference in flexibility of the two languages.

yet I don't hear
people new to perl complaining about this. Perhaps this is because
perl has more to complain about, from a newbie perspective, but I
think it's also because perl gives better syntax error messages. For
example, if you get a runaway unclosed string or regexp (//) operator,
the perl interpreter will, in the syntax error, also say something
like "possible runaway string beginning line NN".

I know that perl does this and I liked that at the time when I used perl. But:

This seems to me like it would be a simple addition to the ruby
parser, to mention the line number of the thing it's trying to close
when expecting tEND, or ']', or some other nested thing that can go
off the end of the file.

IMHO you're greatly underestimating the effor. I guess even for perl it was not "a simple addition" - and from what I can see a lot more efforts were put into perl vs. ruby.

Kind regards

  robert

Perl also has a separate compile step. (It may not seem like it, but
it does, internally.)

-austin

···

On 8/18/06, Daniel Martin <martin@snowplow.org> wrote:

I'll note that perl has a similarly flexible syntax, yet I don't hear
people new to perl complaining about this. Perhaps this is because
perl has more to complain about, from a newbie perspective, but I
think it's also because perl gives better syntax error messages. For
example, if you get a runaway unclosed string or regexp (//) operator,
the perl interpreter will, in the syntax error, also say something
like "possible runaway string beginning line NN".

This seems to me like it would be a simple addition to the ruby
parser, to mention the line number of the thing it's trying to close
when expecting tEND, or ']', or some other nested thing that can go
off the end of the file.

--
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * http://www.halostatue.ca/feed/
               * austin@zieglers.ca

good point. I forgot about them being optional. Of course, the example also proves the point that statement terminators facilitate
better reporting of these types of errors.

David Vallner wrote:

···

Mike Cargal wrote:

So, IMHO, the primary cause is the absence of those pesky statement terminators.

Ruby has a statement terminator, it's just optional. If this is an issue for you, feel free to use it.

Feeding the file:

1 class Foo
2
3 def baz
4 puts 'quux';
5 end
6
7 def foo
8 puts 'bar';
9 return Integer("10";
10 end
11
12 def fred
13 puts 'barney'
14 end
15
16 end

to ruby results in:
      foo.rb:9: syntax error
  foo.rb:16: syntax error

And the parser fails rather cleanly on the first unclosed paren. It's not quite as if the syntax were strict, and I'm sure some things can slip through, but it's a help.

Ruby isn't about enforcing rules and making decisions for you, it's about TIMTOWDI and adopting conventions that help you personally from the options available.

David Vallner

That's what a lint style program would do, not the compiler. Compilers are there to check correctness against the formal language definition, not to make opinions about your code, and quoting James Britt: Ruby assumes the developer is a grown-up. If you're newbly enough to make trivial syntax errors, noone's forcing you to use the language.

If you think the syntax checking in the compiler could be made better, go ahead and hack the parser to do it. But I don't recall this topic being present on the list and complained about any often or so vocally, so I doubt it's critical that effort be spent from the core Ruby developer team in that direction.

Oh, before I forget: Whining gets you nowhere, and trolling belongs to slashdot. Cut that out.

David Vallner

···

On Fri, 18 Aug 2006 09:58:31 +0200, Firstname Surname <rubyforum@wendlink.com> wrote:

the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

"Firstname Surname" <rubyforum@wendlink.com> wrote in message
news:a466c7740a34c06801792de17814b637@example.com...

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

    There's a standard response to comments like this. The code is open
source, so if you really think it's that easy, why don't _you_ do it? It
works better for wikis...

The trouble I had was with a line that had an extra '.' like:
@entry = Entry.find(params[:id].
instead of
@entry = Entry.find(params[:id]

and another error that i think was just a missing bracket maybe like
@entry = Entry.find(params[:id

I've used many different languages with various levels of syntax error
help, but never one this bad. I just wrote a quick script to remove
functions from 'def' to 'end' in order and check the syntax over, thus
quickly finding which function the error is in. This is a lot better
than looking through a couple hundred line controller file. If I can do
this in 10 minutes, I can't help but think it's just laziness on the
part of the syntax checker not to provide at least hints where it thinks
the problem might be.

    Okay, judging from your examples, I think the problem is that you're
hitting a rather sore spot in the Ruby interpreter. I've done a few
examples and it looks as if the Ruby parser does a lazy evaluation of
parenthesized (and bracketed) things. It just so happens that these are
exactly the kind of mistakes you make so they really seem to bite you.
    The only thing I can suggest is that you close all your brackets and
functions before filling them. So, while editing, you'll write things in
this order:

def some_function
end

    ..and then...

def some_method
    @member[]
end

    ...and finally...

def some_method
    @member[@other + 2]
end

I understand that ruby is flexible (I'm not not sure this is a merit
when using for a large web app), but the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

    Well, you're welcome to try Django but you should be warned that the
Python syntax is every bit as flexable as Ruby, so if you really don't
think that's a merit for a large web application...
    Python software is generally more mature than Ruby (probably because
it's much older) but the syntax is less consistent (perhaps because it's
much older?), so it's a trade off. You'll have to decide what's important
to you and go from there...
    Good luck!

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

It'd take more work than you think. See, you've actually got several
issues that you keep repeating in your example code, and that could be
the problem. The use of a high-quality editor and good practices will
also help.

Here's the real trick that you're forgetting that makes it a bit harder
for Ruby to deal with what you're talking about: it is not just
line-oriented, it is expression oriented. Expressions may be terminated
either by an unambiguous line ending (that is, it doesn't end with a
continuation character \, comma, period, or an operator that says that
the next item will be on the next line) or semicolons.

This is perfectly legal:

  abc .
    foo

although it's really bad style.

Remember, though, that:

  abc
  foo

is the same thing as

  abc; foo

So if you have something that can expect the result of an expression
waiting open, you've got a mismatch immediately. More on that in a
moment.

The trouble I had was with a line that had an extra '.' like:
@entry = Entry.find(params[:id].

Note that you've got a syntax error on this line *without* the extra
period. You're missing a period. What this means *very* specifically is
that you're now working on an expression that should ultimately return
something and be closed with a parenthesis. Now, if you *had* a closing
parenthesis, Ruby would give you a syntax error on a multiline
expression like that for a method call (or so it seems with 1.8.2 on my
as-yet default Tiger install).

It would give you the error at the *end* of the code:

    irb(main):030:0> abc.foo(1
    irb(main):031:1> +2
    irb(main):032:1> +3
    irb(main):033:1> +4)
    SyntaxError: compile error
    (irb):31: syntax error
    (irb):33: syntax error
            from (irb):33

So Ruby's smart enough to tell you what's going on as best it can. If
you reach an "end" before your parameter list finishes, it's going to
complain about the "end", unless it's matched, like so:

    irb(main):039:0> abc.foo(1
    irb(main):040:1> end
    SyntaxError: compile error
    (irb):40: syntax error
            from (irb):40
    irb(main):041:0> abc.foo(1
    irb(main):042:1> if
    irb(main):043:2* end
    irb(main):044:1> end
    SyntaxError: compile error
    (irb):42: syntax error
            from (irb):44

As you can see, Ruby is quite a bit smarter than you think -- but it
can't tell that that parenthesis is the start of the problem. I don't
think that a C/C++ compiler could tell that, either.

and another error that i think was just a missing bracket maybe like
@entry = Entry.find(params[:id

Again, [] expects the result of an expression. These two are equivalent:

    params[:id]
    params.[](id)

I've used many different languages with various levels of syntax error
help, but never one this bad.

I personally doubt that. I think that you're expecting a magic bullet
because you're hearing a lot of hype about Rails, and it's not. Believe
me, Ruby's syntax errors may not be the best, but there are two points
to consider toward making them better:

  1. There is no compile step as such. Compiling and execution time are
     the same. This means that:
  2. Increasing the amount of data kept around during parsing to report
     better syntax errors increases the amount of data that must be kept
     in the interpreter for your program and probably not disappear for
     the life of your program.

Can they be better? Yes. But for Ruby to keep track of where each
parenthesis was opened (etc.) and report back that value instead of
where the specific ending token was found that Ruby *does* report on
would require significantly more memory during parsing.

It has been suggested that we might be able to take advantage of a
lint-like tool, but most people who have suggested such have not found
it to be worth the amount of effort as opposed to using a really good
editor with folding and syntax highlighting.

I just wrote a quick script to remove functions from 'def' to 'end' in
order and check the syntax over, thus quickly finding which function
the error is in.

A good editor would help you with that far more than having to write a
quick script.

This is a lot better than looking through a couple hundred line
controller file.

Your controller may be too large, then.

If I can do this in 10 minutes, I can't help but think it's just
laziness on the part of the syntax checker not to provide at least
hints where it thinks the problem might be.

If it doesn't carry that information around, then it can't. And it's not
laziness on the syntax checker's part. I think you're misunderstanding
what's necessary for parsing this sort of error.

I understand that ruby is flexible (I'm not not sure this is a merit
when using for a large web app), but the checker could easily check
based on 'standard' expected syntax and be more helpful about error
location 99% of the time.

Not really. The "standard" expected syntax has changed. Remember, both
of these are legal and equivalent:

    foo = bar :baz
    foo = bar(:baz)

I think you're expecting more than you're going to get from any syntax
checker except a lint-like thing.

-austin

···

On 8/18/06, Firstname Surname <rubyforum@wendlink.com> wrote:
--
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * http://www.halostatue.ca/feed/
               * austin@zieglers.ca

Both Perl and Ruby allow parenthesis to be dropped in most cases:

   $ perl -we 'print("Hello world!\n")'
   Hello world!
   $ perl -we 'print "Hello world!\n"'
   Hello world!

James Edward Gray II

···

On Aug 18, 2006, at 10:10 AM, Robert Klemme wrote:

Daniel Martin wrote:

I'll note that perl has a similarly flexible syntax,

I don't think so. For example, you must terminate statements with a ";" (or did that change in the latest version?) and you cannot omit parentheses.

James Britt: Ruby assumes the
developer is a grown-up. If you're newbly enough to make trivial syntax
errors, noone's forcing you to use the language.

Yeah, that guys sounds like a lot of fun at parties.

Daniel Martin wrote:

Robert Klemme <shortcutter@googlemail.com> writes:

This seems to me like it would be a simple addition to the ruby
parser, to mention the line number of the thing it's trying to close
when expecting tEND, or ']', or some other nested thing that can go
off the end of the file.

Bingo, that sounds like an awesome idea. I've seen the 'possible runaway
expression' type errors in a bunch of compilers starting with Borland
turbo c++ in '93 that used to say things like 'possible unterminated
string on line x' etc. I understand that ruby doesn't have line
termination and is expressions, but I don't get yet why the compiler
couldn't tell you the line where the unterminated expression started.

IMHO you're greatly underestimating the effor. I guess even for perl it
was not "a simple addition" - and from what I can see a lot more efforts
were put into perl vs. ruby.

Would it really be that difficult to say 'unterminated expression
started on line 10' rather than 'syntax error - last line of file - go
fish'?

I get messages like this occasionally. I've found that a good editor is
the most helpful thing in finding these errors. I use TextMate on my
Mac.

Thanks, this is a big help, it took me awhile to figure out the
keystroke for auto indent in Textmate.

···

--
Posted via http://www.ruby-forum.com/.

Firstname Surname a écrit :

Gregor Kopp wrote:

As "Firstname Surname" said, the error messages from the ruby
interpreter are not really significant, they could be more verbose.
Maybe Ruby 2.0 will do it better.
However, a good editor may help, but it can not be the solution, or?

Regards, Gregor

Robert Klemme schrieb:

I think the syntax checker could be a little more verbose, and I don't
think it would take much work.

http://rubyforge.org/projects/rubygrammar/

http://rubyforge.org/pipermail/rubygrammar-grammarians/

···

--
Lionel Thiry

Personal web site: http://users.skynet.be/lthiry/