Parse tree?

I have a situation where it may make sense, although for some reason I
don't want to, to use something like ParseTree to pull out each
complete Ruby expression and pass it to a block which will then
evaluate it in a particular context and make all kinds of magic
happen.

Is this a reasonable ParseTree use case?

@expressions_parse_tree_found.each |exp|
  ArbitraryModule.module_eval(&exp)
end

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you'd
need Ruby2Ruby for that, and even then it might not work.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

why not?

···

On Jan 2, 2008, at 22:01 , Giles Bowkett wrote:

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you'd
need Ruby2Ruby for that, and even then it might not work.

> Sorry, that was a dumb question, PT gives you sexps, not Ruby, you'd
> need Ruby2Ruby for that, and even then it might not work.

why not?

As far as I can tell you get the whole shebang back, and I just need
particular expressions.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

Giles Bowkett wrote:

As far as I can tell you get the whole shebang back, and I just need
particular expressions.
  

Maybe you can use the ruby_parser gem?

···

--
Florian Frank

does that help?

···

On Jan 3, 2008, at 06:46 , Giles Bowkett wrote:

Sorry, that was a dumb question, PT gives you sexps, not Ruby, you'd
need Ruby2Ruby for that, and even then it might not work.

why not?

As far as I can tell you get the whole shebang back, and I just need
particular expressions.

>> pt = ParseTree.new.parse_tree_for_string("1+2")
=> [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
>> Ruby2Ruby.new.process(pt.first)
=> "(1 + 2)"

ryan. even though i've seen that before i thought i'd add: that is insanely cool!

a @ http://codeforpeople.com/

···

On Jan 3, 2008, at 1:10 PM, Ryan Davis wrote:

>> pt = ParseTree.new.parse_tree_for_string("1+2")
=> [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
>> Ruby2Ruby.new.process(pt.first)
=> "(1 + 2)"

does that help?

--
it is not enough to be compassionate. you must act.
h.h. the 14th dalai lama

>>> Sorry, that was a dumb question, PT gives you sexps, not Ruby, you'd
>>> need Ruby2Ruby for that, and even then it might not work.
>>
>> why not?
>
> As far as I can tell you get the whole shebang back, and I just need
> particular expressions.

> >> pt = ParseTree.new.parse_tree_for_string("1+2")
> => [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
> >> Ruby2Ruby.new.process(pt.first)
> => "(1 + 2)"

does that help?

Sort of. It is obviously full of nifty and then some, and I think you
could use it to unit test metaprogramming, which is something the
metaprogramming-averse complain about the absence of.

But say I do this:

pt = ParseTree.new.parse_tree_for_string("1 + 2 ; 1 * 2")

(string):1: warning: useless use of + in void context
=> [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
[:lit, 1], :, [:array, [:lit, 2]]]]]

Ruby2Ruby.new.process(pt.first)

=> "(1 + 2)\n(1 * 2)\n"

What if I just want 1 * 2?

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

It is really cool. I've been using R2R to build an auto-rspec thing
that re-creates the context of an expression so that you can test
against it. It's actually quite fantastic.

Judson

···

On Jan 3, 2008 1:33 PM, ara.t.howard <ara.t.howard@gmail.com> wrote:

On Jan 3, 2008, at 1:10 PM, Ryan Davis wrote:

>
>> >> pt = ParseTree.new.parse_tree_for_string("1+2")
>> => [[:call, [:lit, 1], :+, [:array, [:lit, 2]]]]
>> >> Ruby2Ruby.new.process(pt.first)
>> => "(1 + 2)"
ryan. even though i've seen that before i thought i'd add: that is
insanely cool!

--
Your subnet is currently 169.254.0.0/16. You are likely to be eaten by a grue.

Sort of. It is obviously full of nifty and then some, and I think you

could use it to unit test metaprogramming, which is something the
metaprogramming-averse complain about the absence of.

But say I do this:

pt = ParseTree.new.parse_tree_for_string("1 + 2 ; 1 * 2")

(string):1: warning: useless use of + in void context
=> [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
[:lit, 1], :, [:array, [:lit, 2]]]]]

Ruby2Ruby.new.process(pt.first)

=> "(1 + 2)\n(1 * 2)\n"

What if I just want 1 * 2?

It depends on what sort of '1 * 2' you want... Do you want the last expression in a block/defn/whatever? Like, are you trying to analyze what the return type is? Or do you want to be aware of that void context warning and discard the crap? Or do you simply want the second thingy? Or do you want something entirely different that I am not anticipating?

There are a couple ways to go about it, depending on your answer above. The easiest is Sexp/Array manipulation. Let's say you were interested in the arg list of the last call in the block:

>> sexp = Sexp.from_array pt.first
=> s(:block, s(:call, s(:lit, 1), :+, s(:array, s(:lit, 2))), s(:call, s(:lit, 1), :*, s(:array, s(:lit, 2))))
>> sexp.last
=> s(:call, s(:lit, 1), :*, s(:array, s(:lit, 2)))
>> sexp.last.array
=> s(:array, s(:lit, 2))

This is a bad example because it has two calls in the block. With unique sub-sexp types in a sexp, you can just pull them out by name:

>> sexp.call.array

But it balks when it is ambiguous.

But... if that isn't what you want, there are more powerful means of dealing with stuff:

class CallArgAnalyzer < SexpProcessor
   def process_call sexp
     sexp.shift # :call
     recv = sexp.shift
     name = sexp.shift
     args = sexp.shift

     # do something with args

     return s(:nil) # or whatever... depends on context
   end
end

>> CallArgAnalyzer.new.process(sexp.first)
=> s(:nil)

Granted, this doesn't actually DO anything, but the framework to do something is there and solid... Do note that SexpProcessor is meant for transformational processing so it generally expects a sexp back (you can set what the return type should be for all process_* methods). The real point is that it is very easy to get at what you want and mess with it.

···

On Jan 3, 2008, at 15:50 , Giles Bowkett wrote:

> But say I do this:
>
>>> pt = ParseTree.new.parse_tree_for_string("1 + 2 ; 1 * 2")
> (string):1: warning: useless use of + in void context
> => [[:block, [:call, [:lit, 1], :, [:array, [:lit, 2]]], [:call,
> [:lit, 1], :, [:array, [:lit, 2]]]]]
>>> Ruby2Ruby.new.process(pt.first)
> => "(1 + 2)\n(1 * 2)\n"
>
> What if I just want 1 * 2?

It depends on what sort of '1 * 2' you want... Do you want the last
expression in a block/defn/whatever? Like, are you trying to analyze
what the return type is? Or do you want to be aware of that void
context warning and discard the crap? Or do you simply want the second
thingy? Or do you want something entirely different that I am not
anticipating?

Well, I'm writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up *all* the statements in a
block, the whole thing kinds of goes south. So I have something which
generates all the individual statements perfectly but completely falls
apart when I want to create several statements. So it looks as if my
only option might be to rewrite everything, possibly adding a parser,
or using Ruby2C, or something like that. It's kind of a pain in the
ass.

But if I have code like this:

single_line(statement)
statement(:which => spans,
               :multiple => lines)

And I can turn it into an array of procs, or blocks, or even strings,
then I can just do

code_chunks.each {|code_chunk| @code << Generators.module_eval(code_chunk)}

Or something similar, and harness everything I have so far without
throwing it all away and starting over from scratch or close to it.
All I really need at that point is a collection of code chunks, in
some format which module_eval will be comfortable with.

There are a couple ways to go about it, depending on your answer
above. The easiest is Sexp/Array manipulation. Let's say you were
interested in the arg list of the last call in the block:

> >> sexp = Sexp.from_array pt.first
> => s(:block, s(:call, s(:lit, 1), :+, s(:array, s(:lit, 2))),
> s(:call, s(:lit, 1), :*, s(:array, s(:lit, 2))))
> >> sexp.last
> => s(:call, s(:lit, 1), :*, s(:array, s(:lit, 2)))
> >> sexp.last.array
> => s(:array, s(:lit, 2))

This is a bad example because it has two calls in the block. With
unique sub-sexp types in a sexp, you can just pull them out by name:

> >> sexp.call.array

But it balks when it is ambiguous.

But... if that isn't what you want, there are more powerful means of
dealing with stuff:

class CallArgAnalyzer < SexpProcessor
   def process_call sexp
     sexp.shift # :call
     recv = sexp.shift
     name = sexp.shift
     args = sexp.shift

     # do something with args

     return s(:nil) # or whatever... depends on context
   end
end

> >> CallArgAnalyzer.new.process(sexp.first)
> => s(:nil)

Granted, this doesn't actually DO anything, but the framework to do
something is there and solid... Do note that SexpProcessor is meant
for transformational processing so it generally expects a sexp back
(you can set what the return type should be for all process_*
methods). The real point is that it is very easy to get at what you
want and mess with it.

Very easy is a relative term. :slight_smile:

In practice none of the code I'm working with or expecting to work
with has more than one statement on a line, but obviously the whole
problem is that it has many statements in a block.

The thing is, I posted a few days ago about a problem with if, not
being able to override it, and if I'm using a fully-fledged parsing
library then I might as well undo my kludge and enable real support
for conditionals like if, unless, while, etc. But it seems like more
work than I'd intended. But I can't deny that being able to use real
Ruby in the code which generates the other code would be a serious
improvement.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

Well, I'm writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up *all* the statements in a
block, the whole thing kinds of goes south. So I have something which
generates all the individual statements perfectly but completely falls
apart when I want to create several statements.

Does this help?

>> require 'rubygems' ; require 'ruby2ruby'
=> true
>> def code_for &block; block.to_ruby; end
=> nil
>> code_for do
?> single_line(statement)
>> statement(:which => spans,
?> :multiple => lines)
>> end
=> "proc {\n single_line(statement)\n statement(:which => spans, :multiple => lines)\n}"

There are clean ways to ensure it isn't a proc as well.

···

On Jan 3, 2008, at 21:06 , Giles Bowkett wrote:

From the description you give, it sounds like you could just create a
new block that just calls a list of other blocks, and do ; separation?

Ie, if the line above is in a method called "foo(&block)", call it
with something like

foo do
   my_blocks.collect { |block| block.call + ";" }
end

Probably I'm not understanding your requirements, just thought I'd air
it in case it is this simple.

Eivind.

···

On Jan 4, 2008 6:06 AM, Giles Bowkett <gilesb@gmail.com> wrote:

Well, I'm writing a code generator, and I thought I was very close to
finished, except I was basically doing this:

generated_code = Generators.module_eval(&block)

And that works great for tiny bits of code, it works at the statement
level, but when I want to just collect up *all* the statements in a
block, the whole thing kinds of goes south.

> And that works great for tiny bits of code, it works at the statement
> level, but when I want to just collect up *all* the statements in a
> block, the whole thing kinds of goes south. So I have something which
> generates all the individual statements perfectly but completely falls
> apart when I want to create several statements.

Does this help?

> >> require 'rubygems' ; require 'ruby2ruby'
> => true
> >> def code_for &block; block.to_ruby; end
> => nil
> >> code_for do
> ?> single_line(statement)
> >> statement(:which => spans,
> ?> :multiple => lines)
> >> end
> => "proc {\n single_line(statement)\n statement(:which =>
> spans, :multiple => lines)\n}"

There are clean ways to ensure it isn't a proc as well.

That totally solves my problem. Or at least, ***if*** it's guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven't tried it yet, I've just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

> Well, I'm writing a code generator, and I thought I was very close to
> finished, except I was basically doing this:
>
> generated_code = Generators.module_eval(&block)
>
> And that works great for tiny bits of code, it works at the statement
> level, but when I want to just collect up *all* the statements in a
> block, the whole thing kinds of goes south.

From the description you give, it sounds like you could just create a
new block that just calls a list of other blocks, and do ; separation?

Ie, if the line above is in a method called "foo(&block)", call it
with something like

foo do
   my_blocks.collect { |block| block.call + ";" }
end

Probably I'm not understanding your requirements, just thought I'd air
it in case it is this simple.

The hard part is splitting an existing block into an array of blocks,
or strings, based on the logical division of statements; in other
words to say "turn this block into an array of statements" (in a way
that Ruby understands) is the tricky part.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

What is your 1-liner requirement for?

···

On Jan 4, 2008, at 08:07 , Giles Bowkett wrote:

That totally solves my problem. Or at least, ***if*** it's guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven't tried it yet, I've just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

> That totally solves my problem. Or at least, ***if*** it's guaranteed
> to turn multi-line statements into one-liners every time, that very
> probably totally solves my problem. (I haven't tried it yet, I've just
> proved it to be true, etc.) I can just pull it out of the proc{} with
> a regex and then split it on the newlines. Baddabing baddaboom.

What is your 1-liner requirement for?

Weird synch thing there. I plugged it in and it worked perfectly,
except that I'm splitting blocks now where I don't need to. I can hack
around it by regexing on "do" but it's non perfecto.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com