Parse tree redux

somehow missent

···

---------- Forwarded message ----------

More issues with this. Context in forwarded msg.

Here's a (probably very naive) implementation of the discussed code.
It takes code blocks, turns them into individual statements, and
module_evals each individual statement. It does this by turning blocks
into code with Ruby2Ruby, splitting the code on new lines, and
module-eval()ing each statements in the resulting array.

If you think about it the problem is obvious. Here's the code.

    def generate(&block)
      code = ""
      statements_in_the_block(block).each do |statement|
        code << Generators.module_eval(statement)
      end
      code
    end
    def statements_in_the_block(block)
      statements = /proc \{\n(.+)\}/m.match(block.to_ruby)[1].split(/\n/)
      statements.map {|statement| statement.gsub!(/^\s+/, "")}
      statements.map {|statement| statement.gsub!(/\s+/, " ")}
      statements
    end

The problem, of course, is nested blocks. If you hit a block which
contains another block, you get eval trying to eval block-starting
lines like

eval("block_taking_method do |variable|")

which of course goes bonk.

Probably what I need to do instead (thinking out loud) is pull the
blocks and statements out of the actual s-exps, like have some method
which takes s-exps and then turns them into blocks and statements, and
work from there. Ironically the blocks it's breaking on do fine, due
to a quirk in my code, the inner blocks are good but the larger blocks
break.

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

---------- Forwarded message ----------
From: Giles Bowkett <gilesb@gmail.com>
Date: Jan 4, 2008 8:07 AM
Subject: Re: parse tree?
To: ruby-talk ML <ruby-talk@ruby-lang.org>

> And that works great for tiny bits of code, it works at the statement
> level, but when I want to just collect up *all* the statements in a
> block, the whole thing kinds of goes south. So I have something which
> generates all the individual statements perfectly but completely falls
> apart when I want to create several statements.

Does this help?

> >> require 'rubygems' ; require 'ruby2ruby'
> => true
> >> def code_for &block; block.to_ruby; end
> => nil
> >> code_for do
> ?> single_line(statement)
> >> statement(:which => spans,
> ?> :multiple => lines)
> >> end
> => "proc {\n single_line(statement)\n statement(:which =>
> spans, :multiple => lines)\n}"

There are clean ways to ensure it isn't a proc as well.

That totally solves my problem. Or at least, ***if*** it's guaranteed
to turn multi-line statements into one-liners every time, that very
probably totally solves my problem. (I haven't tried it yet, I've just
proved it to be true, etc.) I can just pull it out of the proc{} with
a regex and then split it on the newlines. Baddabing baddaboom.

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

I still don't understand the 1-line obsession... and yes, that is naive, it'll break on strings, sub-procs, pretty much all blocks, etc. You can't treat "statements" in ruby as each line of a block. #to_ruby is probably not what you want.

···

On Jan 4, 2008, at 14:44 , Giles Bowkett wrote:

If you think about it the problem is obvious. Here's the code.

   def generate(&block)
     code = ""
     statements_in_the_block(block).each do |statement|
       code << Generators.module_eval(statement)
     end
     code
   end
   def statements_in_the_block(block)
     statements = /proc \{\n(.+)\}/m.match(block.to_ruby)[1].split(/\n/)
     statements.map {|statement| statement.gsub!(/^\s+/, "")}
     statements.map {|statement| statement.gsub!(/\s+/, " ")}
     statements
   end

Ryan Davis:
i recently started playing with Parsetree and Ruby2Ruby and Ruby2C, what's bugging me it's compleate lack of wiki/documentation - i read your blog - but it's hard to distinguish stuff that's still related from obsolete.

aside from that, code you wrote is plain amazing :slight_smile:

and i already have few idea's how to use it ^^

I still don't understand the 1-line obsession... and yes, that is
naive, it'll break on strings, sub-procs, pretty much all blocks, etc.
You can't treat "statements" in ruby as each line of a block. #to_ruby
is probably not what you want.

Unfortunately I think that's true. What I need to do basically (I
think) is just rewrite the stuff that generates the code.

···

--
Giles Bowkett

Podcast: http://hollywoodgrit.blogspot.com
Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org
Tumblelog: http://giles.tumblr.com

complete lack of documentation?!? I write developer tools. I basically expect my user base to be able to read code and I try to write the clearest, smallest code possible such that often doco isn't necessary. But in this case I'm a bit confused:

     http://parsetree.rubyforge.org/ParseTree/ !!!

the doco is actually pretty good (coverage wise--I can't judge the content as I don't read it), ~60% according to dcov and it is beating me up on things like my error classes.

···

On Jan 5, 2008, at 00:03 , Marcin Raczkowski wrote:

i recently started playing with Parsetree and Ruby2Ruby and Ruby2C, what's bugging me it's compleate lack of wiki/documentation - i read your blog - but it's hard to distinguish stuff that's still related from obsolete.

aside from that, code you wrote is plain amazing :slight_smile:

and i already have few idea's how to use it ^^

complete lack of documentation?!? I write developer tools. I basically expect my user base to be able to read code and I try to write the clearest, smallest code possible such that often doco isn't necessary. But in this case I'm a bit confused:

    http://parsetree.rubyforge.org/ParseTree/ !!!

the doco is actually pretty good (coverage wise--I can't judge the content as I don't read it), ~60% according to dcov and it is beating me up on things like my error classes.

Hmmm. Problem is that sometimes when you look for a solution for certain problem it's nice if you have short introduction on how to use it.

If you have to evaluate 3-5 solutions and you have to get to know each and every one of them it's really time consuming - short summary is not enought

Documentation itself is good - true, but it took me few hours of reading your blog, wikipedia and source code to understand what was basic concept and how i could use it to make my idea work :slight_smile:

not to mention that someone without solid c background and idea how ruby internals work won't be able to understand jack.
I finally got the grasp when i stared inspecting Ruby2Ruby.

and bugs that I reported for Ruby2C were never even acknowledged :slight_smile:

Aside from that this stuff is really powerfull, and having it when working on FreeRIDE coloring and code completion (which i never finished) would be awesome.

My idea of transfering ruby code beetween procesing nodes might come true. And i would like to really thank you for that :slight_smile:

greets