For-in vs. map closures

ruby-talk
1

I was experimenting with closures and JavaScript's and Ruby's
behavior. Ruby seems to close over its arguments automatically if
using map:

[1, 2, 3].map { |n| lambda { n * n } }.map { |b| b.call() } # [1, 4,
9]

But using "for" iteration, returns a different result:

a = []
for n in [1, 2, 3] do a << lambda { n * n } end
a.map { |b| b.call() } # [9, 9, 9]

This is also the behavior in JavaScript, which frequently catches
developers off guard when attaching event handlers to elements.

Can anyone explain the difference? Is map creating a new anonymous
function every iteration?

Mike

2

I don't know, it seems like very peculiar behaviour. It does create new
lambdas every time, but strangely, the objects created within the block are
shared.

a = Array.new
for s in %w(a b c)
  s2 = s
  a << lambda { [ s2 , s , s2.object_id ] }
end

a.map { |b| b.call() } # => [["c", "c", 2148175880], ["c", "c",
2148175880], ["c", "c", 2148175880]]
a[0].object_id # => 2148175860
a[1].object_id # => 2148175840
a[2].object_id # => 2148175820

···

On Wed, May 5, 2010 at 1:35 AM, Mike Austin <mike_ekim@yahoo.com> wrote:

I was experimenting with closures and JavaScript's and Ruby's
behavior. Ruby seems to close over its arguments automatically if
using map:

[1, 2, 3].map { |n| lambda { n * n } }.map { |b| b.call() } # [1, 4,
9]

But using "for" iteration, returns a different result:

a =
for n in [1, 2, 3] do a << lambda { n * n } end
a.map { |b| b.call() } # [9, 9, 9]

This is also the behavior in JavaScript, which frequently catches
developers off guard when attaching event handlers to elements.

Can anyone explain the difference? Is map creating a new anonymous
function every iteration?

Mike

3

I was experimenting with closures and JavaScript's and Ruby's
behavior. Ruby seems to close over its arguments automatically if
using map:

[1, 2, 3].map { |n| lambda { n * n } }.map { |b| b.call() } # [1, 4,
9]

Here, n is local to the block, so the n in the lambda is different
every time. The lambda is a closure over the scope of the block, and
in that block the variable n is different in each iteration.

But using "for" iteration, returns a different result:

a =
for n in [1, 2, 3] do a << lambda { n * n } end
a.map { |b| b.call() } # [9, 9, 9]

Here n is the same in all iterations (it's outside the do ... end),
for defines it in the scope that is above the block, so it's the same
for all iterations. The lambda closes over that scope, but the n in
all lambdas point to the same object. So after the loop, n is 3, and
so all lambdas see 3 as the value.

Jesus.

···

On Wed, May 5, 2010 at 8:35 AM, Mike Austin <mike_ekim@yahoo.com> wrote:

4

Hi --

I was experimenting with closures and JavaScript's and Ruby's
behavior. Ruby seems to close over its arguments automatically if
using map:

[1, 2, 3].map { |n| lambda { n * n } }.map { |b| b.call() } # [1, 4,
9]

But using "for" iteration, returns a different result:

a =
for n in [1, 2, 3] do a << lambda { n * n } end
a.map { |b| b.call() } # [9, 9, 9]

This is also the behavior in JavaScript, which frequently catches
developers off guard when attaching event handlers to elements.

Can anyone explain the difference? Is map creating a new anonymous
function every iteration?

Mike

I don't know, it seems like very peculiar behaviour. It does create new
lambdas every time, but strangely, the objects created within the block are
shared.

a = Array.new
for s in %w(a b c)
s2 = s
a << lambda { [ s2 , s , s2.object_id ] }
end

a.map { |b| b.call() } # => [["c", "c", 2148175880], ["c", "c",
2148175880], ["c", "c", 2148175880]]
a[0].object_id # => 2148175860
a[1].object_id # => 2148175840
a[2].object_id # => 2148175820

for doesn't create a new local scope:

defined?(b)

=> nil

for a in [1]; b = 2; end

=> [1]

b

=> 2

It's similar to if and friends in this respect. So s and s2 in your
example are the same variables every time through the loop.

David

···

On Wed, 5 May 2010, Josh Cheek wrote:

On Wed, May 5, 2010 at 1:35 AM, Mike Austin <mike_ekim@yahoo.com> wrote:

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

THE Ruby training with Black/Brown/McAnally
     COMPLEAT Coming to Chicago area, June 18-19, 2010!
              RUBYIST http://www.compleatrubyist.com

5

Wow, those were both really good explanations. Thanks David and Jesus.

6

Wow, those were both really good explanations. Thanks David and Jesus.

In fact there was something not so clear (or just plain wrong,
depending who you ask :-), in my explanation. I said:

Here n is the same in all iterations (it's outside the do ... end),
for defines it in the scope that is above the block

The fact that n is outside the do...end has nothing to do with the
rest of the explanation. As David said, for doesn't start a new scope,
so even if it were like this it wouldn't yield different results:

irb(main):008:0> a =
=>
irb(main):009:0> for i in [1,2] do
irb(main):010:1> value = i
irb(main):011:1> a << lambda {value * value}
irb(main):012:1> end
=> [1, 2]
irb(main):013:0> a.each {|l| puts l.call}
4
4

Even though value is inside the block, as the for keyword doesn't
create a new scope, value is defined in the outer scope. So, what's
here between the do...end is not a regular ruby block, in fact, the do
is optional:

irb(main):008:0> a =
=>
irb(main):009:0> for i in [1,2]
irb(main):010:1> value = i
irb(main):011:1> a << lambda {value * value}
irb(main):012:1> end
=> [1, 2]
irb(main):013:0> a.each {|l| puts l.call}
4
4

After either of those:

irb(main):020:0> defined? value
=> "local-variable"

Jesus.

···

On Wed, May 5, 2010 at 5:45 PM, Josh Cheek <josh.cheek@gmail.com> wrote:

7

So if a new context is created for every block call in map(), in
theory it should be slower than a for-in loop, correct? I'll have to
try it when I get home.

Mike

···

On May 5, 9:01 am, Jesús Gabriel y Galán <jgabrielyga...@gmail.com> wrote:

On Wed, May 5, 2010 at 5:45 PM, Josh Cheek <josh.ch...@gmail.com> wrote:
> Wow, those were both really good explanations. Thanks David and Jesus.

In fact there was something not so clear (or just plain wrong,
depending who you ask :-), in my explanation. I said:

> Here n is the same in all iterations (it's outside the do ... end),
> for defines it in the scope that is above the block

The fact that n is outside the do...end has nothing to do with the
rest of the explanation. As David said, for doesn't start a new scope,
so even if it were like this it wouldn't yield different results:

irb(main):008:0> a =
=>
irb(main):009:0> for i in [1,2] do
irb(main):010:1> value = i
irb(main):011:1> a << lambda {value * value}
irb(main):012:1> end
=> [1, 2]
irb(main):013:0> a.each {|l| puts l.call}
4
4

Even though value is inside the block, as the for keyword doesn't
create a new scope, value is defined in the outer scope. So, what's
here between the do...end is not a regular ruby block, in fact, the do
is optional:

irb(main):008:0> a =
=>
irb(main):009:0> for i in [1,2]
irb(main):010:1> value = i
irb(main):011:1> a << lambda {value * value}
irb(main):012:1> end
=> [1, 2]
irb(main):013:0> a.each {|l| puts l.call}
4
4

After either of those:

irb(main):020:0> defined? value
=> "local-variable"

Jesus.

8

I suppose it's slower but I think it's mainly because the block call.
Let us know if you test it.

Jesus.

···

On Wed, May 5, 2010 at 11:50 PM, Mike Austin <mike_ekim@yahoo.com> wrote:

On May 5, 9:01 am, Jesús Gabriel y Galán <jgabrielyga...@gmail.com> > wrote:

On Wed, May 5, 2010 at 5:45 PM, Josh Cheek <josh.ch...@gmail.com> wrote:
> Wow, those were both really good explanations. Thanks David and Jesus.

In fact there was something not so clear (or just plain wrong,
depending who you ask :-), in my explanation. I said:

> Here n is the same in all iterations (it's outside the do ... end),
> for defines it in the scope that is above the block

The fact that n is outside the do...end has nothing to do with the
rest of the explanation. As David said, for doesn't start a new scope,
so even if it were like this it wouldn't yield different results:

irb(main):008:0> a =
=>
irb(main):009:0> for i in [1,2] do
irb(main):010:1> value = i
irb(main):011:1> a << lambda {value * value}
irb(main):012:1> end
=> [1, 2]
irb(main):013:0> a.each {|l| puts l.call}
4
4

Even though value is inside the block, as the for keyword doesn't
create a new scope, value is defined in the outer scope. So, what's
here between the do...end is not a regular ruby block, in fact, the do
is optional:

irb(main):008:0> a =
=>
irb(main):009:0> for i in [1,2]
irb(main):010:1> value = i
irb(main):011:1> a << lambda {value * value}
irb(main):012:1> end
=> [1, 2]
irb(main):013:0> a.each {|l| puts l.call}
4
4

After either of those:

irb(main):020:0> defined? value
=> "local-variable"

Jesus.

So if a new context is created for every block call in map(), in
theory it should be slower than a for-in loop, correct? I'll have to
try it when I get home.

9

No not really (at least for 1.9)

here is the output
ruby -v loops.rb
ruby 1.9.1p378 (2010-01-10 revision 26273) [i686-linux]
Rehearsal ----------------------------------------
loop 1.340000 0.000000 1.340000 ( 1.508669)
each 1.340000 0.010000 1.350000 ( 1.388928)
------------------------------- total: 2.690000sec

           user system total real
loop 1.380000 0.010000 1.390000 ( 1.419883)
each 1.370000 0.000000 1.370000 ( 1.417870)

ruby -v loops.rb
ruby 1.8.7 (2009-06-12 patchlevel 174) [i486-linux]
Rehearsal ----------------------------------------
loop 3.690000 0.650000 4.340000 ( 4.927702)
each 4.570000 0.770000 5.340000 ( 5.764859)
------------------------------- total: 9.680000sec

           user system total real
loop 4.000000 0.630000 4.630000 ( 4.739408)
each 4.510000 0.680000 5.190000 ( 5.489487)

My naïve interpretation would be that the closure is here anyway only
that with for in it contains a ref to i and with each a copy. Memory
usage should be up though.

Cheers
R.

···

On Wed, May 5, 2010 at 11:50 PM, Mike Austin <mike_ekim@yahoo.com> wrote:

So if a new context is created for every block call in map(), in
theory it should be slower than a for-in loop, correct? I'll have to
try it when I get home.

Mike

--
The best way to predict the future is to invent it.
-- Alan Kay

10

And adding a third benchmark, using map like the original

under ruby 1.9.2

Rehearsal ----------------------------------------
loop 1.100000 0.140000 1.240000 ( 1.500455)
each 1.060000 0.120000 1.180000 ( 1.430910)
map 1.060000 0.080000 1.140000 ( 1.315434)
------------------------------- total: 3.560000sec

           user system total real
loop 1.090000 0.090000 1.180000 ( 1.327022)
each 1.210000 0.100000 1.310000 ( 1.491797)
map 1.080000 0.080000 1.160000 ( 1.288349)

and under 1.8.7

Rehearsal ----------------------------------------
loop 2.950000 0.620000 3.570000 ( 4.199053)
each 3.900000 0.210000 4.110000 ( 4.849201)
map 4.610000 0.110000 4.720000 ( 5.622664)
------------------------------ total: 12.400000sec

           user system total real
loop 3.620000 0.200000 3.820000 ( 4.475057)
each 4.760000 0.100000 4.860000 ( 5.561921)
map 4.150000 0.400000 4.550000 ( 5.249561)
ruby for_vs_each.rb 24.21s user 1.69s system 85% cpu 30.320 total

I would note that, since the loop case gives different results, and if
those results aren't the droids you are looking for, then the
performance doesn't really matter. If I don't need to get the correct
results, I can write arbitrarily fast code <G>.

I almost never use for in Ruby, it makes me feel more like I'm writing
C or BASIC, or Algol, or even FORTRAN.

I always keep in mind that the order is 1. Make it run, 2. Make it
right, 3. Make it fast enough, 4. Make it small enough.

1 and 2 are mandatory, 3 and 4 depend on what enough means.

···

On Thu, May 6, 2010 at 5:17 AM, Robert Dober <robert.dober@gmail.com> wrote:

On Wed, May 5, 2010 at 11:50 PM, Mike Austin <mike_ekim@yahoo.com> wrote:

So if a new context is created for every block call in map(), in
theory it should be slower than a for-in loop, correct? I'll have to
try it when I get home.

Mike

No not really (at least for 1.9)
bench.rb · GitHub
here is the output
ruby -v loops.rb
ruby 1.9.1p378 (2010-01-10 revision 26273) [i686-linux]
Rehearsal ----------------------------------------
loop 1.340000 0.000000 1.340000 ( 1.508669)
each 1.340000 0.010000 1.350000 ( 1.388928)
------------------------------- total: 2.690000sec

      user     system      total        real

loop 1.380000 0.010000 1.390000 ( 1.419883)
each 1.370000 0.000000 1.370000 ( 1.417870)

ruby -v loops.rb
ruby 1.8.7 (2009-06-12 patchlevel 174) [i486-linux]
Rehearsal ----------------------------------------
loop 3.690000 0.650000 4.340000 ( 4.927702)
each 4.570000 0.770000 5.340000 ( 5.764859)
------------------------------- total: 9.680000sec

      user     system      total        real

loop 4.000000 0.630000 4.630000 ( 4.739408)
each 4.510000 0.680000 5.190000 ( 5.489487)

--
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Github: rubyredrick (Rick DeNatale) · GitHub
Twitter: @RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

11

<snip> I almost never use for in Ruby, it makes me feel more like I'm writing

C or BASIC, or Algol, or even FORTRAN.

Ah, eventually I understand what "fornever" means.

I always keep in mind that the order is 1. Make it run, 2. Make it
right, 3. Make it fast enough, 4. Make it small enough.

1 and 2 are mandatory, 3 and 4 depend on what enough means.

I feel 4's importance is largely underestimated.

Cheers
R.

···

On Thu, May 6, 2010 at 2:51 PM, Rick DeNatale <rick.denatale@gmail.com> wrote:

--
The best way to predict the future is to invent it.
-- Alan Kay