Eval/binding question

ruby-talk
1

I tried to create local variables from a name=>value hash passed as a parameter, but I can't get it to work. The following test program shows what is happening:

    def test1
      eval "x=25"
      eval 'print "x=#{x}\n"'
    end

    test1

    def test2
      eval "x=25"
      print "x=#{x}\n"
    end

    test2

produces:

    x=25
    test.rb:189:in `test2': undefined local variable or method `x' for
    main:Object ( NameError)
            from test.rb:192

It seems that changes to local variables are not reflected back to the executing environment.

Does anyone out there know how to solve this problem?

-- stefan

2

"Stefan Kaes" <skaes@gmx.net> schrieb im Newsbeitrag
news:4236E882.3050602@gmx.net...

I tried to create local variables from a name=>value hash passed as a
parameter, but I can't get it to work. The following test program shows
what is happening:

    def test1
      eval "x=25"
      eval 'print "x=#{x}\n"'
    end

    test1

    def test2
      eval "x=25"
      print "x=#{x}\n"
    end

    test2

produces:

    x=25
    test.rb:189:in `test2': undefined local variable or method `x' for
    main:Object ( NameError)
            from test.rb:192

It seems that changes to local variables are not reflected back to the
executing environment.

Does anyone out there know how to solve this problem?

You can't. Local variables are determined during parse phase and you
cannot access local variables that you create via eval other than via
eval. Consider

16:11:59 [robert.klemme]: ruby -e 'eval "x=10";puts x'
-e:1: undefined local variable or method `x' for main:Object (NameError)
16:13:51 [robert.klemme]: ruby -e 'x=nil;eval "x=10";puts x'
10
16:13:57 [robert.klemme]: ruby -e 'eval "x=10";eval "puts x"'
10

IOW, you can manipulate predefined locals but not those that spring into
existence during an eval.

Kind regards

    robert

3

Hi,

I tried to create local variables from a name=>value hash passed as a
parameter, but I can't get it to work. The following test program shows
what is happening:

local variables should be determined at compile time, thus local
variables defined first in the eval'ed string, can only be accessed from
other eval'ed strings. In addition, they will be more ephemeral in
Ruby2, so that these variables will not be accessed from outside.

In summary, I recommend you not to use local variables for your
purpose. They are wrong tool for it.

              matz.

···

In message "Re: eval/binding question" on Tue, 15 Mar 2005 23:02:44 +0900, Stefan Kaes <skaes@gmx.net> writes:

4

Yukihiro Matsumoto wrote:

Hi,

I tried to create local variables from a name=>value hash passed as a parameter, but I can't get it to work. The following test program shows what is happening:

local variables should be determined at compile time, thus local
variables defined first in the eval'ed string, can only be accessed from
other eval'ed strings. In addition, they will be more ephemeral in
Ruby2, so that these variables will not be accessed from outside.

Well, I don't agree. From a language designers point of view x=5 and eval "x=5" should do the same thing: modify the current binding by introducing a new value-binding with name x and value 5. I don't know of any language which behaves differently (e.g. LISP works like this AFAIK). Of course, as a compiler writer, one might prefer to be able to determine all local variables by looking at the source. But this is just a wish to make the compiler simpler or enable better optimization.

In summary, I recommend you not to use local variables for your
purpose. They are wrong tool for it.

            matz.

Maybe I should explain why I tried this:

when using ERB one can compile a template into source, which can then be eval'ed using eval. Local variables for use in the template can be set up using the sort of eval given in my example. However, in this case the 'compiled' template code gets reparsed on each evaluation of the template code. I wanted to speed up this process by defining a function (eval "def fun; #{src}; end", aBinding), thereby pasring the code only once. This works pretty well and gives about 25% increase in speed. The performance gain will be much bigger, once ruby gets a real JIT. But, as it turned out, local variables are not accessible when the defined function is executed.

I saw two solutions: First, change the local variables to instance variables on the class executing the function. This works, but does not feel right (name space pollution). Second, put all locals in an instance hash and set up local functions inside the defined function from the hash. Which does not work.

Any other ideas?

···

In message "Re: eval/binding question" > on Tue, 15 Mar 2005 23:02:44 +0900, Stefan Kaes <skaes@gmx.net> writes:

5

Hi,

local variables should be determined at compile time, thus local
variables defined first in the eval'ed string, can only be accessed from
other eval'ed strings. In addition, they will be more ephemeral in
Ruby2, so that these variables will not be accessed from outside.

Well, I don't agree. From a language designers point of view x=5 and
eval "x=5" should do the same thing: modify the current binding by
introducing a new value-binding with name x and value 5. I don't know of
any language which behaves differently (e.g. LISP works like this
AFAIK). Of course, as a compiler writer, one might prefer to be able to
determine all local variables by looking at the source. But this is just
a wish to make the compiler simpler or enable better optimization.

You don't have to agree here. Each language designer have different
point of view. I'm not going to change my mind by your "should".

when using ERB one can compile a template into source, which can then be
eval'ed using eval. Local variables for use in the template can be set
up using the sort of eval given in my example. However, in this case the
'compiled' template code gets reparsed on each evaluation of the
template code. I wanted to speed up this process by defining a function
(eval "def fun; #{src}; end", aBinding), thereby pasring the code only
once. This works pretty well and gives about 25% increase in speed. The
performance gain will be much bigger, once ruby gets a real JIT. But, as
it turned out, local variables are not accessible when the defined
function is executed.

I'm not sure I'm fully understand you. Can you show us your code?

              matz.

···

In message "Re: eval/binding question" on Wed, 16 Mar 2005 01:17:55 +0900, Stefan Kaes <skaes@gmx.net> writes:

6

Maybe I should explain why I tried this:

when using ERB one can compile a template into source, which can then

be eval'ed using eval. Local variables for use in the template can be

set

hash and set up local functions inside the defined function from the

Well, that's funny :slight_smile:

I just finished working on a VERY similar bit of code.

FYI, I came up with a sligthly altered version of your first solution.
In my case, name space polution is not a problem.

The code will be available soon as part of Nitro's new Mailer
subsystem.

regards,
-g.

ps:

···

--
http://nitro.rubyforge.org

7

ERb already does this for you. See ERB::Defmethod. Unfortunately, there are no docs for it, but you can dig around in Rails to see how it is used.

PGP.sig (186 Bytes)

···

On 15 Mar 2005, at 08:17, Stefan Kaes wrote:

when using ERB one can compile a template into source, which can then be eval'ed using eval. Local variables for use in the template can be set up using the sort of eval given in my example. However, in this case the 'compiled' template code gets reparsed on each evaluation of the template code. I wanted to speed up this process by defining a function (eval "def fun; #{src}; end", aBinding), thereby pasring the code only once.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

8

Yukihiro Matsumoto wrote:

I'm not sure I'm fully understand you. Can you show us your code?

Yes, but I need to clean up first.

-- stefan

9

code attached.

In the real environment, the names of the locally assigned variables can not be determined in advance, of course.

evalbinding.rb (714 Bytes)

test.rhtml (130 Bytes)

10

Yukihiro Matsumoto wrote:

You don't have to agree here. Each language designer have different
point of view. I'm not going to change my mind by your "should".

Let me rephrase: the semantics of eval should be defined in a way that holds no surprises. I would expect that writing

    ...
    eval "code"
    ...

and

    ...
    code
    ...

should be identical.

-- stefan

11

Ok, I lied. ERB::DefMethod is both not used by Rails (does something else) but it is also unsuitable for the task.

Anyhow, as my penance, here's my implementation, I did use def_method, though:

http://segment7.net/projects/ruby/snippets/erb_cache.rb

PGP.sig (186 Bytes)

···

On 15 Mar 2005, at 19:55, Eric Hodel wrote:

On 15 Mar 2005, at 08:17, Stefan Kaes wrote:

when using ERB one can compile a template into source, which can then be eval'ed using eval. Local variables for use in the template can be set up using the sort of eval given in my example. However, in this case the 'compiled' template code gets reparsed on each evaluation of the template code. I wanted to speed up this process by defining a function (eval "def fun; #{src}; end", aBinding), thereby pasring the code only once.

ERb already does this for you. See ERB::Defmethod. Unfortunately, there are no docs for it, but you can dig around in Rails to see how it is used.

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

12

Eric Hodel wrote:

···

On 15 Mar 2005, at 08:17, Stefan Kaes wrote:

when using ERB one can compile a template into source, which can then be eval'ed using eval. Local variables for use in the template can be set up using the sort of eval given in my example. However, in this case the 'compiled' template code gets reparsed on each evaluation of the template code. I wanted to speed up this process by defining a function (eval "def fun; #{src}; end", aBinding), thereby pasring the code only once.

ERb already does this for you. See ERB::Defmethod. Unfortunately, there are no docs for it, but you can dig around in Rails to see how it is used.

That's not true. ERB::Defmethod does not work this way. And Rails uses the less efficient method, which I wanted to improve on (and did). And I wanted the local varaiable solution mainly for backwards compatibility with rendering partials in Rails. But the question about eval/binding semantics was posted to this list because I thought (and still think) that the semantics of eval is not what it should be.

13

Hi,

Let me rephrase: the semantics of eval should be defined in a way that
holds no surprises.

I know what you mean. But it's not good strategy to persuade me to
use "your expectation" or "your surprise". This "limitation" has a
lot of good aspects, such as better performance, better error
detection, etc. I'd love to pay the cost of small restriction for
these benefits as a language designer.

OK, I think this is enough. Let us focus on solving "your problem".

              matz.

···

In message "Re: eval/binding question" on Wed, 16 Mar 2005 03:29:53 +0900, Stefan Kaes <skaes@gmx.net> writes:

14

Hi,

In the real environment, the names of the locally assigned variables can
not be determined in advance, of course.

I'm sorry to say you can't pre-compile eRB allowing local variable set
not determined in advance. I know allowing this will make your
program 25% faster, but this also would make any other programs
slower.

If you have limited set of local variables, you can do something like:

  require 'erb'

  class Test
    def initialize(path, *lv)
      @erb = ERB.new(File.read(path), nil, '%')
      @locals = {}
      evalstr = "def run_erb\n" +
        lv.collect{|k| "#{k} = @locals[\"#{k}\"]\n"}.join +
        "#{@erb.src}\n_erbout\nend\n"
      print evalstr, "\n\n"
      eval evalstr, nil, path, 0
    end
    def values(hash)
      @locals.update(hash)
      hash.each {|k,v|
        self.instance_variable_set("@#{k}", v)
      }
    end
  end

  t = Test.new("/tmp/test.rhtml", 'mumu')
  t.values("mumu" => "HELLO")
  print t.run_erb

              matz.

···

In message "Re: eval/binding question" on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <skaes@gmx.net> writes:

15

Eric Hodel wrote:

Ok, I lied. ERB::DefMethod is both not used by Rails (does something else) but it is also unsuitable for the task.

Why did you lie???

Anyhow, as my penance, here's my implementation, I did use def_method, though:

http://segment7.net/projects/ruby/snippets/erb_cache.rb

Thanks for the code, but I already knew this would also be a possible solution. But probably less efficient than using instance variables, though I have not checked that yet. Did you?

16

+1

I have on a couple of occasions also been bit by this. It would be much
nicer (to the programmer, at any rate) for
    code and eval "code"
to behave consistently with each other.

In particular, manipulating variable bindings is extremely difficult
without.

So I'd like to revive this request, if possible ...

17

Yukihiro Matsumoto wrote:

Hi,

In the real environment, the names of the locally assigned variables can not be determined in advance, of course.

I'm sorry to say you can't pre-compile eRB allowing local variable set
not determined in advance. I know allowing this will make your
program 25% faster, but this also would make any other programs
slower.

I fail to see why it would make other code slower. I am convinced with a proper compiler implementation there would'nt be any speed difference for local variables introduced via explicit assignment. Access to predeclared variables would use an optimized access method, whereas access to variable names whose declaration point cannot be determined by statical program analysis would use a less efficient access method.

If you have limited set of local variables, you can do something like:

require 'erb'

class Test
   def initialize(path, *lv)
     @erb = ERB.new(File.read(path), nil, '%')
     @locals = {}
     evalstr = "def run_erb\n" +
       lv.collect{|k| "#{k} = @locals[\"#{k}\"]\n"}.join +
       "#{@erb.src}\n_erbout\nend\n"
     print evalstr, "\n\n"
     eval evalstr, nil, path, 0
   end
   def values(hash)
     @locals.update(hash)
     hash.each {|k,v|
       self.instance_variable_set("@#{k}", v)
     }
   end
end

t = Test.new("/tmp/test.rhtml", 'mumu')
t.values("mumu" => "HELLO")
print t.run_erb

            matz.

That's simlar to the way I "solved" "my problem" already. Thanks.

-- stefan

···

In message "Re: eval/binding question" > on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <skaes@gmx.net> writes:

18

Eric Hodel wrote:

Ok, I lied. ERB::DefMethod is both not used by Rails (does something else) but it is also unsuitable for the task.

Why did you lie???

I didn't bother to double-check first. My bad.

Anyhow, as my penance, here's my implementation, I did use def_method, though:

http://segment7.net/projects/ruby/snippets/erb_cache.rb

Thanks for the code, but I already knew this would also be a possible solution. But probably less efficient than using instance variables, though I have not checked that yet. Did you?

attr* methods are optimized for speed so you don't have all the overhead of a regular method call, but I doubt this will be faster than ivars.

An ivar implementation would be pretty easy too, use instance_variable_set instead of send.

Here's how Ruby optimizes attr* methods:

$ parse_tree_show
class Example
   attr_reader :x
   def y() @y; end
end
[[:class,
   :Example,
   :Object,
   [:defn, :x, [:ivar, :@x]],
   [:defn, :y, [:scope, [:block, [:args], [:ivar, :@y]]]]]]

No scope setup is done for attr*.

PGP.sig (186 Bytes)

···

On 15 Mar 2005, at 21:23, Stefan Kaes wrote:

--
Eric Hodel - drbrain@segment7.net - http://segment7.net
FEC2 57F1 D465 EB15 5D6E 7C11 332A 551C 796C 9F04

19

"Stefan Kaes" <skaes@gmx.net> schrieb im Newsbeitrag
news:4237CBC1.9070209@gmx.net...

Yukihiro Matsumoto wrote:

>Hi,
>
>
>>In the real environment, the names of the locally assigned variables

can

>>not be determined in advance, of course.
>
>I'm sorry to say you can't pre-compile eRB allowing local variable set
>not determined in advance. I know allowing this will make your
>program 25% faster, but this also would make any other programs
>slower.
>
>
I fail to see why it would make other code slower. I am convinced with a
proper compiler implementation there would'nt be any speed difference
for local variables introduced via explicit assignment. Access to
predeclared variables would use an optimized access method, whereas
access to variable names whose declaration point cannot be determined by
statical program analysis would use a less efficient access method.

I wonder in which cases this would be useful. This code is artificial,
since one would never write test2 like you did:

    def test2
      eval "x=25"
      print "x=#{x}\n"
    end

If you write something similar but with a string coming from somewhere
else

    def test2(s)
      eval s
      print "x=#{x}\n"
    end

you know already that "s" must contain an assignment to "x". So it's not
a problem to predeclare it IMHO.

I don't really see which real world problems would be solved with a more
complex lookup for locals, which would make the second variant succeed
without the explicit declaration of "x".

Kind regards

    robert

···

>In message "Re: eval/binding question" > > on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <skaes@gmx.net> writes:

20

Robert Klemme wrote:

"Stefan Kaes" <skaes@gmx.net> schrieb im Newsbeitrag
news:4237CBC1.9070209@gmx.net...

Yukihiro Matsumoto wrote:

Hi,

In the real environment, the names of the locally assigned variables
     

can

not be determined in advance, of course.

I'm sorry to say you can't pre-compile eRB allowing local variable set
not determined in advance. I know allowing this will make your
program 25% faster, but this also would make any other programs
slower.

I fail to see why it would make other code slower. I am convinced with a
proper compiler implementation there would'nt be any speed difference
for local variables introduced via explicit assignment. Access to
predeclared variables would use an optimized access method, whereas
access to variable names whose declaration point cannot be determined by
statical program analysis would use a less efficient access method.
   
I wonder in which cases this would be useful. This code is artificial,
since one would never write test2 like you did:

   def test2
     eval "x=25"
     print "x=#{x}\n"
   end

This code is artificial because I only wrote it to demonstrate the semantics of the current implementation.

If you write something similar but with a string coming from somewhere
else

   def test2(s)
     eval s
     print "x=#{x}\n"
   end

you know already that "s" must contain an assignment to "x". So it's not
a problem to predeclare it IMHO.

For this simple demonstration program you are absolutely right: there is no reason to write it that way, other than to demonstrate current semantics.

I don't really see which real world problems would be solved with a more
complex lookup for locals, which would make the second variant succeed
without the explicit declaration of "x".

Kind regards

   robert

The current semantics introduces an artificial barrier between code executed in *eval "code"* vs. just writing *code*, which makes the semantics more difficult to explain and understand. Ideally, these two forms should have identical semantics. As I have already explained, the complexity of variable access increases only for variables whose declaration point cannot be determined statically and therfore would not have a negative performance impact on most programs.

The real world example can be found in Rails. It has a function render_partial, which takes an erb template name (tn), a value (v) and a name=>value hash (local_assigns). It makes the value v accessible as a local variable named "tn" and all name=>value pairs in local_assigns accessible by their name. This works, because the whole template source is evaluated using an eval, but requires reparsing of the erb-code each time the template gets interpreted. And, of course, the code that implements this function cannot know the names of the local varaiables in advance.

As soon as one tries to abstract the erb code into a function the whole scheme breaks down, because there is no way to dynamically declare local variables. I solved this problem by resorting to instance variables. That is okay for me, I just rewrote my template code, but it breaks backwards compatibility. So this valuable optimisation cannot be included in Rails, unless everyone rewrites their templates. For this reason, I have not proposed it to the Rails community.

regards, stefan

···

In message "Re: eval/binding question" >>> on Wed, 16 Mar 2005 02:45:41 +0900, Stefan Kaes <skaes@gmx.net> >>> >>> >writes: