Can anyone explain to me what Yuya’s package bRuby
http://bruby.sourceforge.jp is about?
-austin
– Austin Ziegler, austin@halostatue.ca on 2002.10.31 at 21.40.10
Well, since none of our native Japanese speakers have responded yet,
I’ll attempt a superficial explanation. If you can read Japanese at all,
you may have as good an idea as I do. But anyway:
“bRuby is an extension library for dumping or un-dumping the
internal tree structure of a Ruby interpreter.”
Apparently it is being developed in connection with the Japan
Information Technology Promotion Agency’s [“Unexplored Youth” software
creation project]*, which seems to be a sort of contest (?) for young
developers. This year’s theme is something about “Ruby script hiding and
interchange with Windows executables” (don’t ask me what those two have
to do with each other.
I think I’ll stop before I really put my foot in my mouth.
- That’s my literal translation. The Japanese probably has an idiomatic
meaning that makes more sense.
···
On Fri, Nov 01, 2002 at 11:40:51AM +0900, Austin Ziegler wrote:
Can anyone explain to me what Yuya’s package bRuby
http://bruby.sourceforge.jp is about?
–
Matt Gushee When a nation follows the Way,
Englewood, Colorado, USA Horses bear manure through
mgushee@havenrock.com its fields;
http://www.havenrock.com/ When a nation ignores the Way,
Horses bear soldiers through
its streets.
--Lao Tzu (Peter Merel, trans.)
In “Re: bRuby?”
Can anyone explain to me what Yuya’s package bRuby
http://bruby.sourceforge.jp is about?Well, since none of our native Japanese speakers have responded yet,
I’ll attempt a superficial explanation. If you can read Japanese at all,
you may have as good an idea as I do. But anyway:“bRuby is an extension library for dumping or un-dumping the
internal tree structure of a Ruby interpreter.”
Right. It seems that it is a library for
dumping/un-dumping ruby’s internal syntax tree.
Apparently it is being developed in connection with the Japan
Information Technology Promotion Agency’s [“Unexplored Youth” software
creation project]*, which seems to be a sort of contest (?) for young
developers. This year’s theme is something about “Ruby script hiding and
interchange with Windows executables” (don’t ask me what those two have
to do with each other.
It might be translated like “Ruby script encryption and
conversion to Windows executable”.
“Mito-youth” is not exactly “Unexplored Youth”, but, well,
“Funding young hackers” project run by IPA.
···
on Fri, 1 Nov 2002 13:06:55 +0900 Matt Gushee mgushee@havenrock.com wrote:
On Fri, Nov 01, 2002 at 11:40:51AM +0900, Austin Ziegler wrote:
Hisashi Morita
In article 20021107151904lBE13F@ohmsha.co.jp,
In “Re: bRuby?”
Can anyone explain to me what Yuya’s package bRuby
http://bruby.sourceforge.jp is about?Well, since none of our native Japanese speakers have responded yet,
I’ll attempt a superficial explanation. If you can read Japanese at all,
you may have as good an idea as I do. But anyway:“bRuby is an extension library for dumping or un-dumping the
internal tree structure of a Ruby interpreter.”Right. It seems that it is a library for
dumping/un-dumping ruby’s internal syntax tree.
Yet another Ruby parser… Let’s see, there’s:
Ripper: Index of /archive/ripper
Ruth: RubyVM - Browse Files at SourceForge.net
and now bRuby.
All three of these seem to modify Matz’s Ruby Yacc file and create an
extension from the resulting c file.
Are there other Ruby parsers out there?
Phil
···
hmorita hmorita@ohmsha.co.jp wrote:
on Fri, 1 Nov 2002 13:06:55 +0900 > Matt Gushee mgushee@havenrock.com wrote:
On Fri, Nov 01, 2002 at 11:40:51AM +0900, Austin Ziegler wrote:
There’s JRuby http://jruby.sourceforge.net/, a pure Java
implementation of the Ruby interpreter.
Xandy
···
On Thu, 2002-11-07 at 03:28, Phil Tomson wrote:
Yet another Ruby parser… Let’s see, there’s:
Ripper: http://i.loveruby.net/archive/ripper/
Ruth: RubyVM - Browse Files at SourceForge.netand now bRuby.
All three of these seem to modify Matz’s Ruby Yacc file and create an
extension from the resulting c file.Are there other Ruby parsers out there?
%% Are there other Ruby parsers out there?
I am working on a parser - which I’m dubbing “Parse-R” - that will run as a
standalone shared library and that is not limited to being used as a Ruby
Extension. In fact, direct hooks into the Ruby runtime are not a feature of
my parser, on purpose; what I’m aiming for a is a generic Ruby parser that
allows you to supply the necessary low-level hooks into the parser to do
something interesting. The goal is to create a standard Ruby parser that can
be used as the front end to many different tools - the interpreter, a source
browser, a once-and-future byte-code compiler, etc.
I started working in earnest again on it yesterday, as a matter of fact, and
I hope to have a first release very soon. I know I have been saying this for
a while… but I renewed my efforts to work on it yesterday (my birthday,
not coincidentally), after having endured many challenges to spending any
significant time on it. One of those have gone away, and measurable progress
is expected in the near-term, like within a week or two.
I have been conspicuously long on promises and short on results for this and
other Ruby related projects to which I have long wanted to contribute. That
will change, now that my situation has changed.
Sincerely,
Bob
%% -----Original Message-----
%% From: Phil Tomson [mailto:ptkwt@shell1.aracnet.com]
%% Sent: Thursday, November 07, 2002 3:28 AM
%% To: ruby-talk ML
%% Subject: Re: bRuby? (Yet another Ruby parser)
%%
%%
%% In article 20021107151904lBE13F@ohmsha.co.jp,
···
%% hmorita hmorita@ohmsha.co.jp wrote:
%% >In “Re: bRuby?”
%% > on Fri, 1 Nov 2002 13:06:55 +0900
%% > Matt Gushee mgushee@havenrock.com wrote:
%% >
%% >| On Fri, Nov 01, 2002 at 11:40:51AM +0900, Austin Ziegler wrote:
%% >| > Can anyone explain to me what Yuya’s package bRuby
%% >| > http://bruby.sourceforge.jp is about?
%% >|
%% >| Well, since none of our native Japanese speakers have responded yet,
%% >| I’ll attempt a superficial explanation. If you can read
%% Japanese at all,
%% >| you may have as good an idea as I do. But anyway:
%% >|
%% >| “bRuby is an extension library for dumping or un-dumping the
%% >| internal tree structure of a Ruby interpreter.”
%% >
%% >Right. It seems that it is a library for
%% >dumping/un-dumping ruby’s internal syntax tree.
%%
%% Yet another Ruby parser… Let’s see, there’s:
%% Ripper: http://i.loveruby.net/archive/ripper/
%% Ruth: http://sourceforge.net/project/showfiles.php?group_id=26000
%%
%% and now bRuby.
%%
%% All three of these seem to modify Matz’s Ruby Yacc file and create an
%% extension from the resulting c file.
%%
%%
%% Phil
%%
%%
%%
In article aqd5u501rtt@enews2.newsguy.com,
ptkwt@shell1.aracnet.com (Phil Tomson) writes:
Are there other Ruby parsers out there?
I made yet another Ruby parser:
http://cvs.m17n.org/cgi-bin/viewcvs/xmlize/?cvsroot=ruby
It’s a standalone command which markup Ruby source as XML.
It uses parse.y’s production rules but no actions i.e. no more
lex_state! The lexer implemented from scratch in flex.
Note that it’s undocumented.
···
–
Tanaka Akira
In article aqd5u501rtt@enews2.newsguy.com,
ptkwt@shell1.aracnet.com (Phil Tomson) writes:Are there other Ruby parsers out there?
I made yet another Ruby parser:
I’m at work on an LL-1 scanner-parser for my cocorb
tool (it will create an extension).
Not so simple … 
-mark.
In article NCEJJNLDMEJLEJHKNGNHOEKLFDAA.robert.calco@verizon.net,
%% Are there other Ruby parsers out there?
I am working on a parser - which I’m dubbing “Parse-R” - that will run as a
standalone shared library and that is not limited to being used as a Ruby
Extension. In fact, direct hooks into the Ruby runtime are not a feature of
my parser, on purpose; what I’m aiming for a is a generic Ruby parser that
allows you to supply the necessary low-level hooks into the parser to do
something interesting. The goal is to create a standard Ruby parser that can
be used as the front end to many different tools - the interpreter, a source
browser, a once-and-future byte-code compiler, etc.I started working in earnest again on it yesterday, as a matter of fact, and
I hope to have a first release very soon. I know I have been saying this for
a while… but I renewed my efforts to work on it yesterday (my birthday,
not coincidentally), after having endured many challenges to spending any
significant time on it. One of those have gone away, and measurable progress
is expected in the near-term, like within a week or two.I have been conspicuously long on promises and short on results for this and
other Ruby related projects to which I have long wanted to contribute. That
will change, now that my situation has changed.Sincerely,
Bob
%% >Right. It seems that it is a library for
%% >dumping/un-dumping ruby’s internal syntax tree.
%%
%% Yet another Ruby parser… Let’s see, there’s:
%% Ripper: Index of /archive/ripper
%% Ruth: RubyVM - Browse Files at SourceForge.net
%%
%% and now bRuby.
%%
%% All three of these seem to modify Matz’s Ruby Yacc file and create an
%% extension from the resulting c file.
Yeah, I knew about Bob’s project.
I guess the point of my question is that there seem to be at least six
projects out there that are essentially doing the same thing. Aren’t we
duplicating a lot of effort? Is there any way we can work together on one
of these parsers and possibly get results quicker?
Bob,
Did you take a look at Ruth and Ripper before you started your project?
If so how would you say what you are planning is different from those
projects?
Phil
···
Bob Calco robert.calco@verizon.net wrote:
Phil Tomson wrote:
%%
%% All three of these seem to modify Matz’s Ruby Yacc file and create an
%% extension from the resulting c file.Yeah, I knew about Bob’s project.
I guess the point of my question is that there seem to be at least six
projects out there that are essentially doing the same thing. Aren’t we
duplicating a lot of effort? Is there any way we can work
together on one
of these parsers and possibly get results quicker?Bob,
Did you take a look at Ruth and Ripper before you started your project?
If so how would you say what you are planning is different from those
projects?Phil
Bob give his own answer, but since I work closely with Bob (others) on the
FreeRIDE and WxRuby projects I can tell you what I know.
The original motivation for this parser was the FreeRIDE project. The
FreeRIDE IDE needs a fast and accurate Ruby parser to enable some of the
high-end stuff that we want to implement (we are currently using ripper, but
it is using a modified version of the Ruby 1.6 grammar). The goal here is to
put a wrapper around the actual parser that is used by the Ruby interpreter.
The wrapper will have the kind of api that we need and will always be
accurate.
Curt
In article aqecmv12rtd@enews4.newsguy.com,
ptkwt@shell1.aracnet.com (Phil Tomson) writes:
I guess the point of my question is that there seem to be at least six
projects out there that are essentially doing the same thing. Aren’t we
duplicating a lot of effort? Is there any way we can work together on one
of these parsers and possibly get results quicker?
Ruth calls Ruby’s internal parser.
Ripper reuse parse.y including the lexer but not actions.
My xmlize reuse production rules from parse.y but not actions and the
lexer.
Maybe, Mark’s parser for cocorb don’t reuse production rules parse.y because
the rules are not LL(1).
The hardest problem for writing Ruby parser is cloning the heuristics
of lex_state to resolve ambiguity especially with “do” and “{”. I
tried to understand the heuristics but I couldn’t. So xmlize behaves
differently to the original parser in subtle cases.
So, I think Ruth and Ripper are right direction if perfect
compatibility is required. However Ruth may have some restrictions
because internal parser rip off some information such as comments,
exact token position, etc.
The ultimate answer is Rite, of course. It should have reusable
parser component. When it is available, we don’t need to update
various parsers to follow matz’s. But it seems that matz doesn’t
designed the API for the component yet. So we need to propose
suitable API of the component. I think various parser projects may
help matz.
···
–
Tanaka Akira
Phil:
My parser will take the idea of separation from the runtime to its logical
conclusion and provide a very flexible API for attaching the wires and plugs
to whatever backend might need to walk the AST, for whatever purpose.
I have in mind back ends like interpreters, VMs, debuggers, source browsers,
etc. But the API will allow for different levels of depth in terms of
inspecting the tree, so relatively simple utilities like a doc generator
that works from comments only, and therefore that don’t need all the bells
and whistles can ignore the features they don’t need and it will still work.
In other words, my parser will be a generic lexing/parsing front end to any
tool that expects source conforming to Ruby syntax for input. It will expose
itself through a C/C++ API using standard calls so other languages that
support cdecl like Delphi/Kylix can use the DLL/SO file for whatever
purpose.
Implementation wise, Tanaka Akira’s approach, which I saw today for the
first time and is found at
http://cvs.m17n.org/cgi-bin/viewcvs/xmlize/?cvsroot=ruby
is the closest to mine of those available (Ruth, Ripper), though Tanaka’s
goal is an XML representation of the code, which is somewhat different than
mine.
My goal is to have a parser with an API that easy enough to use so that
Tanaka could create the XMLizer utility very quickly using Ruby DL for
instance instead of having to do all the painful work Tanaka had to do (and
did very well, I might add).
Concern about duplication of effort is, I think, valid, but the evils
thereof may be somewhat overrated. I would be happy to merge my efforts with
anybody else, but I’d also be happy just having time to finish what I
started working on myself. I think having parallel efforts until such time
as a consensus emerges about what is really needed and how it really ought
to work is achieved is a good thing. But eventually these efforts converge
as competing approaches are tried and the community gravitates toward a
common goal. I’m pretty laizzez faire about such processes, but willing to
work with anybody who is interested in doing what I’m doing.
Phil, you more than anybody I think is moving the debate about a general
parsing utility in the right direction - you’ve pinged me a few times to
keep moving forward (and I appreciate that), and I might never had had the
pleasure of noticing Tanaka’s work, which gave me a few other ideas I hadn’t
considered even though overall we are taking a very similar approach toward
somewhat different goals.
So - thank you for getting the issue out in the open.
Sincerely,
Bob Calco
%% -----Original Message-----
%% From: freeride-devel-admin@nongnu.org
%% [mailto:freeride-devel-admin@nongnu.org]On Behalf Of Curt Hibbs
%% Sent: Thursday, November 07, 2002 3:32 PM
%% To: ruby-talk@ruby-lang.org; Freeride-Devel
%% Subject: [FR-devel] RE: bRuby? (Yet another Ruby parser)
%%
%%
%% Phil Tomson wrote:
%% > >%%
%% > >%% All three of these seem to modify Matz’s Ruby Yacc file
%% and create an
%% > >%% extension from the resulting c file.
%% >
%% >
%% >
%% > Yeah, I knew about Bob’s project.
%% >
%% > I guess the point of my question is that there seem to be at least six
%% > projects out there that are essentially doing the same thing.
%% Aren’t we
%% > duplicating a lot of effort? Is there any way we can work
%% > together on one
%% > of these parsers and possibly get results quicker?
%% >
%% > Bob,
%% >
%% > Did you take a look at Ruth and Ripper before you started your project?
%% > If so how would you say what you are planning is different from those
%% > projects?
%% >
%% > Phil
%%
%% Bob give his own answer, but since I work closely with Bob
%% (others) on the
%% FreeRIDE and WxRuby projects I can tell you what I know.
%%
%% The original motivation for this parser was the FreeRIDE project. The
%% FreeRIDE IDE needs a fast and accurate Ruby parser to enable some of the
%% high-end stuff that we want to implement (we are currently using
%% ripper, but
%% it is using a modified version of the Ruby 1.6 grammar). The
%% goal here is to
%% put a wrapper around the actual parser that is used by the Ruby
%% interpreter.
%% The wrapper will have the kind of api that we need and will always be
%% accurate.
%%
%% Curt
%%
%%
%%
%% _______________________________________________
%% Freeride-devel mailing list
%% Freeride-devel@nongnu.org
%% http://mail.nongnu.org/mailman/listinfo/freeride-devel
%%
In article hvo65v87ofi.fsf@flux.a02.aist.go.jp,
···
Tanaka Akira akr@m17n.org wrote:
In article aqecmv12rtd@enews4.newsguy.com,
ptkwt@shell1.aracnet.com (Phil Tomson) writes:I guess the point of my question is that there seem to be at least six
projects out there that are essentially doing the same thing. Aren’t we
duplicating a lot of effort? Is there any way we can work together on one
of these parsers and possibly get results quicker?Ruth calls Ruby’s internal parser.
Ripper reuse parse.y including the lexer but not actions.
My xmlize reuse production rules from parse.y but not actions and the
lexer.
Maybe, Mark’s parser for cocorb don’t reuse production rules parse.y because
the rules are not LL(1).The hardest problem for writing Ruby parser is cloning the heuristics
of lex_state to resolve ambiguity especially with “do” and “{”. I
tried to understand the heuristics but I couldn’t. So xmlize behaves
differently to the original parser in subtle cases.So, I think Ruth and Ripper are right direction if perfect
compatibility is required. However Ruth may have some restrictions
because internal parser rip off some information such as comments,
exact token position, etc.
That’s fine for my applications - for example: as a parser for generating
Parrot bytecode. It would of course be a problem for FreeRIDE or ruby
documentation packages.
Phil
It doesn’t use parse.y, rather it is an attempt to see if
I can define ruby in LL(1). I am beginning to think that
it is not possible due to the ambiguities. I still have
a few things to try before I give up …
-mark.
···
At 12:04 PM 11/8/2002 +0900, Tanaka Akira wrote:
Maybe, Mark’s parser for cocorb don’t reuse production rules parse.y because
the rules are not LL(1).
In article NCEJJNLDMEJLEJHKNGNHAEPJFDAA.robert.calco@verizon.net,
Phil:
My parser will take the idea of separation from the runtime to its logical
conclusion and provide a very flexible API for attaching the wires and plugs
to whatever backend might need to walk the AST, for whatever purpose.I have in mind back ends like interpreters, VMs, debuggers, source browsers,
etc. But the API will allow for different levels of depth in terms of
inspecting the tree, so relatively simple utilities like a doc generator
that works from comments only, and therefore that don’t need all the bells
and whistles can ignore the features they don’t need and it will still work.In other words, my parser will be a generic lexing/parsing front end to any
tool that expects source conforming to Ruby syntax for input. It will expose
itself through a C/C++ API using standard calls so other languages that
support cdecl like Delphi/Kylix can use the DLL/SO file for whatever
purpose.Implementation wise, Tanaka Akira’s approach, which I saw today for the
first time and is found athttp://cvs.m17n.org/cgi-bin/viewcvs/xmlize/?cvsroot=ruby
is the closest to mine of those available (Ruth, Ripper), though Tanaka’s
goal is an XML representation of the code, which is somewhat different than
mine.My goal is to have a parser with an API that easy enough to use so that
Tanaka could create the XMLizer utility very quickly using Ruby DL for
instance instead of having to do all the painful work Tanaka had to do (and
did very well, I might add).Concern about duplication of effort is, I think, valid, but the evils
thereof may be somewhat overrated. I would be happy to merge my efforts with
anybody else, but I’d also be happy just having time to finish what I
started working on myself.
I don’t believe that parallel projects are intrinsically evil, mostly I’m
carping about the fact that there are so many ‘half-finished’ efforts. So
I’m thinking that more borrowing could be going on instead of reinvention
in order to perhaps speed things up…
I think having parallel efforts until such time
as a consensus emerges about what is really needed and how it really ought
to work is achieved is a good thing. But eventually these efforts converge
as competing approaches are tried and the community gravitates toward a
common goal. I’m pretty laizzez faire about such processes, but willing to
work with anybody who is interested in doing what I’m doing.Phil, you more than anybody I think is moving the debate about a general
parsing utility in the right direction - you’ve pinged me a few times to
keep moving forward (and I appreciate that), and I might never had had the
pleasure of noticing Tanaka’s work, which gave me a few other ideas I hadn’t
considered even though overall we are taking a very similar approach toward
somewhat different goals.
Yeah, I was aware of what you’re working on, I asked the question here in
this forum so that others who are working on the same sorts of things can
see the answers 
Sorry for being a nag on this issue, but I see having a Ruby parser as
important to some projects I’m working on (Cardinal, RHDL) as well as for
other projects (FreeRIDE, various documentation projects, and various
Ruby-In-Ruby projects).
- A parser derived from Matz’s parse.y file (as your project, Ripper and
Ruth do). - A pure Ruby implementation (as Mark Leidl’s (sp?) is - it’s in the
ruby cvs under ruby-parser/ ).
So - thank you for getting the issue out in the open.
You’re welcome.
So when are you going to have some code to post?
Phil
···
Bob Calco robert.calco@verizon.net wrote:
From my perspective I think we need two types of Ruby parsers:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/51715
···
On Fri, Nov 08, 2002 at 11:45:37PM +0900, Mark Probert wrote:
At 12:04 PM 11/8/2002 +0900, Tanaka Akira wrote:
Maybe, Mark’s parser for cocorb don’t reuse production rules parse.y
because
the rules are not LL(1).It doesn’t use parse.y, rather it is an attempt to see if
I can define ruby in LL(1). I am beginning to think that
it is not possible due to the ambiguities. I still have
a few things to try before I give up …
–
_ _
__ __ | | ___ _ __ ___ __ _ _ __
'_ \ /| __/ __| '__ \ / ` | ’ \
) | (| | |__ \ | | | | | (| | | | |
.__/ _,|_|/| || ||_,|| |_|
Running Debian GNU/Linux Sid (unstable)
batsman dot geo at yahoo dot com
Linux: Where Don’t We Want To Go Today?
– Submitted by Pancrazio De Mauro, paraphrasing some well-known sales talk