To_proc and Proc/block conversion with &

ruby-talk
1

map requires a block, right? So this works:

   words = %w(ardvark anteater llama)

   arr = words.map { |x| x + "!" }
   arr #--> [ardvark!, anteater!, llama!]

But this doesn't:

   arr2 = words.map lambda{ |x| x + "!"} #--> ArgumentError: wrong number of arguments (1 for 0)

We can trigger a call to some object's to_proc method with the ampersand. For example, this
will trigger the symbol's to_proc method, assuming there is one:

   upcase_words = words.map(&:upcase)

and if we have already have this:

   class Symbol
     def to_proc
       lambda{ |x| x.send(self) }
     end
   end

then we get:
   #--> [ARDVARK, ANTEATER, LLAMA]

But, the to_proc method here is returning a Proc, not a block! So why does this work?
As far as I can tell, the ampersand must be doing 2 things:
1- triggering the call to to_proc
2- converting the returned Proc object into a block for map to handle.

Is this right?

2

Hi --

map requires a block, right? So this works:

words = %w(ardvark anteater llama)

arr = words.map { |x| x + "!" }
arr #--> [ardvark!, anteater!, llama!]

But this doesn't:

arr2 = words.map lambda{ |x| x + "!"} #--> ArgumentError: wrong number of arguments (1 for 0)

We can trigger a call to some object's to_proc method with the ampersand. For example, this
will trigger the symbol's to_proc method, assuming there is one:

upcase_words = words.map(&:upcase)

and if we have already have this:

class Symbol
  def to_proc
    lambda{ |x| x.send(self) }
  end
end

then we get:
#--> [ARDVARK, ANTEATER, LLAMA]

But, the to_proc method here is returning a Proc, not a block! So why does this work?

There's no such thing as returning a block. A block is a syntactic
construct. You can, however, have a Proc object in block position,
playing the block role, courtesy of &. (We don't have a really perfect
term for that, I think.)

As far as I can tell, the ampersand must be doing 2 things:
1- triggering the call to to_proc
2- converting the returned Proc object into a block for map to handle.

Is this right?

Yes; the & means: take what's to the right of it (which can be any
expression), evaluate it, call to_proc on the resulting object, and
use the return value of to_proc to play the code block role.

In the case of a lambda, to_proc just returns the receiver. So you can
do:

   array.map &lambda {|x| x * 10 }

Without the &, you're just putting an argument in method-argument
position. Using a Proc object as a method argument does not trigger
any special or block-related behavior. That comes entirely from the
ampersand.

David

···

On Sun, 5 Jul 2009, Russ McBride wrote:

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

3

Russ McBride wrote:

map requires a block, right? So this works:

  words = %w(ardvark anteater llama)

  arr = words.map { |x| x + "!" }
  arr #--> [ardvark!, anteater!, llama!]

But this doesn't:

  arr2 = words.map lambda{ |x| x + "!"} #--> ArgumentError: wrong number of arguments (1 for 0)

Hi, Russ,

It may be helpful to think of the & in

   words.map(&something)

not as an operator (in your example, you have demonstrated why that picture is wrong), but as a way to access a special "slot", different from the positional arguments, for passing an argument of a special kind. The only kind of object the slot can contain is a Proc (and ruby makes an effort to convert the passed object to Proc, using, um, #to_proc).

Using this slot is mutually exclusive with having a _syntactic block_ in the calling context:

   words.map do...end

But from the perspective of the #map method's implementation, these two cases are functionally equivalent. If you were implementing #map, you could access the caller's code in either of the above cases by any of these means:

constructing a Proc using &:

   def map(&b)
     b.call ...
   end

or equivalently with Proc.new:

   def map
     b = Proc.new
     b.call
   end

or (somewhat differently) by yielding:

   def map
     yield ...
   end

Note that the performance is generally better if you avoid constructing Proc objects (assuming you don't need to store them somewhere between method calls).[1] In other words, avoid & and use yield. But this isn't usually significant.

[1] https://groups.google.com/group/ruby-talk-google/msg/e464fd85eb82b3b6

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

4

Hi --

···

On Sun, 5 Jul 2009, Joel VanderWerf wrote:

Russ McBride wrote:

map requires a block, right? So this works:

  words = %w(ardvark anteater llama)

  arr = words.map { |x| x + "!" }
  arr #--> [ardvark!, anteater!, llama!]

But this doesn't:

  arr2 = words.map lambda{ |x| x + "!"} #--> ArgumentError: wrong number of arguments (1 for 0)

Hi, Russ,

It may be helpful to think of the & in

words.map(&something)

not as an operator (in your example, you have demonstrated why that picture is wrong), but as a way to access a special "slot", different from the positional arguments, for passing an argument of a special kind. The only kind of object the slot can contain is a Proc (and ruby makes an effort to convert the passed object to Proc, using, um, #to_proc).

Using this slot is mutually exclusive with having a _syntactic block_ in the calling context:

words.map do...end

But from the perspective of the #map method's implementation, these two cases are functionally equivalent. If you were implementing #map, you could access the caller's code in either of the above cases by any of these means:

constructing a Proc using &:

def map(&b)
   b.call ...
end

or equivalently with Proc.new:

def map
   b = Proc.new
   b.call
end

That's not going to access the caller's code, though, since you're
creating a totally new Proc.

David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

5

Yes; the & means: take what's to the right of it (which can be any
expression), evaluate it, call to_proc on the resulting object, and
use the return value of to_proc to play the code block role.

In the case of a lambda, to_proc just returns the receiver. So you can
do:

   array.map &lambda {|x| x * 10 }

It might be interesting to note that in Ruby1.9 the to_proc is not
called on Proc objects, I wonder what happened in 1.8

536/37 > ruby -rprofile -e '[1].map &Proc::new{|x| x+1}'
  % cumulative self self total
time seconds seconds calls ms/call ms/call name
  0.00 0.00 0.00 1 0.00 0.00 BasicObject#initialize
  0.00 0.00 0.00 1 0.00 0.00 Proc#new
  0.00 0.00 0.00 1 0.00 0.00 Fixnum#+
  0.00 0.00 0.00 1 0.00 0.00 Array#map
  0.00 0.01 0.00 1 0.00 10.00 #toplevel

537/38 > ruby -rprofile -e '[1].map &lambda{|x| x+1}'
  % cumulative self self total
time seconds seconds calls ms/call ms/call name
  0.00 0.00 0.00 1 0.00 0.00 Kernel.lambda
  0.00 0.00 0.00 1 0.00 0.00 Fixnum#+
  0.00 0.00 0.00 1 0.00 0.00 Array#map
  0.00 0.01 0.00 1 0.00 10.00 #toplevel

  ruby -rprofile -e '[1].map(&:succ)'
  % cumulative self self total
time seconds seconds calls ms/call ms/call name
  0.00 0.00 0.00 1 0.00 0.00 Kernel.proc

0.00 0.00 0.00 1 0.00 0.00 Symbol#to_proc<

  0.00 0.00 0.00 1 0.00 0.00 Fixnum#succ
  0.00 0.00 0.00 1 0.00 0.00 Array#map
  0.00 0.01 0.00 1 0.00 10.00 #toplevel

Cheers
Robert

···

On 7/5/09, David A. Black <dblack@rubypal.com> wrote:
--
Toutes les grandes personnes ont d’abord été des enfants, mais peu
d’entre elles s’en souviennent.

All adults have been children first, but not many remember.

[Antoine de Saint-Exupéry]

6

Hi --

···

On Sun, 5 Jul 2009, Robert Dober wrote:

On 7/5/09, David A. Black <dblack@rubypal.com> wrote:

Yes; the & means: take what's to the right of it (which can be any
expression), evaluate it, call to_proc on the resulting object, and
use the return value of to_proc to play the code block role.

In the case of a lambda, to_proc just returns the receiver. So you can
do:

   array.map &lambda {|x| x * 10 }

It might be interesting to note that in Ruby1.9 the to_proc is not
called on Proc objects, I wonder what happened in 1.8

I guess it's an optimization. So take my explanation as behavioral
rather than implementation-faithful.

David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

7

Hi,

Russ McBride wrote:

constructing a Proc using &:

def map(&b)
   b.call ...
end

or equivalently with Proc.new:

def map
   b = Proc.new
   b.call
end

That's not going to access the caller's code, though, since you're
creating a totally new Proc.

Sorry, but these are not the same:

  Proc.new
  Proc.new { }

Try this:

  def f ; Proc.new ; end
  p = f { "hello" }
  p.call
    #=> "hello"

Bertram

···

Am Sonntag, 05. Jul 2009, 19:51:28 +0900 schrieb David A. Black:

On Sun, 5 Jul 2009, Joel VanderWerf wrote:

--
Bertram Scharpf
Stuttgart, Deutschland/Germany
http://www.bertram-scharpf.de

8

Also, it seems to be the same in 1.8. So I guess a refinement of the
description would be: given &expr, if expr is a Proc object, use it as
the block; if not, call to_proc on it.

I don't know whether that's a language spec or just an implementation
detail. I vaguely hope the latter, though I suppose that something
like this:

   func = lambda {|x| x * 10 }
   def func.to_proc
     # some other proc
   end

would be extremely rare....

David

···

On Sun, 5 Jul 2009, David A. Black wrote:

Hi --

On Sun, 5 Jul 2009, Robert Dober wrote:

On 7/5/09, David A. Black <dblack@rubypal.com> wrote:

Yes; the & means: take what's to the right of it (which can be any
expression), evaluate it, call to_proc on the resulting object, and
use the return value of to_proc to play the code block role.

In the case of a lambda, to_proc just returns the receiver. So you can
do:

   array.map &lambda {|x| x * 10 }

It might be interesting to note that in Ruby1.9 the to_proc is not
called on Proc objects, I wonder what happened in 1.8

I guess it's an optimization. So take my explanation as behavioral
rather than implementation-faithful.

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

9

Hi --

···

On Sun, 5 Jul 2009, Bertram Scharpf wrote:

Hi,

Am Sonntag, 05. Jul 2009, 19:51:28 +0900 schrieb David A. Black:

On Sun, 5 Jul 2009, Joel VanderWerf wrote:

Russ McBride wrote:

constructing a Proc using &:

def map(&b)
   b.call ...
end

or equivalently with Proc.new:

def map
   b = Proc.new
   b.call
end

That's not going to access the caller's code, though, since you're
creating a totally new Proc.

Sorry, but these are not the same:

Proc.new
Proc.new { }

Try this:

def f ; Proc.new ; end
p = f { "hello" }
p.call
   #=> "hello"

How odd. I wonder what the point of that is. I'd rather be warned if I
write Proc.new without a block, than have it default to the code
block.

But you're right that I'm wrong in correcting Joel. Rewind.

David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

10

Absolutely agree, but maybe this should go into rubyspec...
and sorry for hijacking.
Cheers
Robert

···

On 7/5/09, David A. Black <dblack@rubypal.com> wrote:

On Sun, 5 Jul 2009, David A. Black wrote:

I don't know whether that's a language spec or just an implementation
detail. I vaguely hope the latter, though I suppose that something
like this:

   func = lambda {|x| x * 10 }
   def func.to_proc
     # some other proc
   end

would be extremely rare....

--
Toutes les grandes personnes ont d’abord été des enfants, mais peu
d’entre elles s’en souviennent.

All adults have been children first, but not many remember.

[Antoine de Saint-Exupéry]

11

David A. Black wrote:

How odd. I wonder what the point of that is. I'd rather be warned if I
write Proc.new without a block, than have it default to the code
block.

Maybe it should be deprecated in practice. It's not apparent what's going on, and rdoc can't pick it up (I presume?). It's usually better to be more explicit using 'def foo(&b)' or yield.

The only use I can think of for this feature[1] is if you want to conditionally instantiate the Proc, as in:

   def defer time
     if time < Time.now
       yield
     else
       pr = Proc.new
       # store pr somewhere and schedule it for execution
     end
   end

[1] ri Proc.new
-------------------------------------------------------------- Proc::new
      Proc.new {|...| block } => a_proc
      Proc.new => a_proc

      From Ruby 1.8

···

------------------------------------------------------------------------
      Creates a new Proc object, bound to the current context. Proc::new
      may be called without a block only within a method with an
      attached block, in which case that block is converted to the Proc
      object.

         def proc_from
           Proc.new
         end
         proc = proc_from { "hello" }
         proc.call #=> "hello"

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

12

Thanks. Interesting discussion.

given &expr, if expr is a Proc object, use it as
the block; if not, call to_proc on it.

... and then use the resulting proc as a block.

Maybe the best way to think of & is a a proc/block converter, which,
if called on something that is neither looks first for a to_proc method before
performing its conversion.

The following seems quite surprising to me and I, too, would rather have
a warning I think than this default behavior.
    def f ; Proc.new ; end
    p = f { "hello" }
    p.call
      #=> "hello"

Maybe it should be deprecated in practice. It's not apparent what's going on, and rdoc can't pick it up (I presume?). It's usually better to be more explicit using 'def foo(&b)' or yield.

The only use I can think of for this feature[1] is if you want to conditionally instantiate the Proc, as in:

def defer time
   if time < Time.now
     yield
   else
     pr = Proc.new
     # store pr somewhere and schedule it for execution
   end
end

But, you could just do something like this, right?

  def defer(time, &some_block)
    if time < Time.now
      yield # or some_block.call
    else
      pr = some_block
      # store pr somewhere and schedule it for execution
    end
  end

Cheers,
Russ

P.S. Joel, I seen now why you were inclined toward Rhodes. :slight_smile:

13

Hi --

···

On Mon, 6 Jul 2009, Russ McBride wrote:

Thanks. Interesting discussion.

given &expr, if expr is a Proc object, use it as
the block; if not, call to_proc on it.

... and then use the resulting proc as a block.

Maybe the best way to think of & is a a proc/block converter, which,
if called on something that is neither looks first for a to_proc method before performing its conversion.

I'm still not sure whether the lack of calling to_proc on the object
if it's already a Proc is just an optimization. I suppose I like the
uniformity of &obj always meaning: call obj.to_proc, but if the
interpreter is optimizing that away I guess I shouldn't cling to it
:slight_smile:

David

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

14

Russ McBride wrote:

Thanks. Interesting discussion.

given &expr, if expr is a Proc object, use it as
the block; if not, call to_proc on it.

... and then use the resulting proc as a block.

Maybe the best way to think of & is a a proc/block converter, which,
if called on something that is neither looks first for a to_proc method before
performing its conversion.

That still makes & sound like an operator method that can be called on an object to produce a different object, though. Blocks are syntactic entities, Procs are semantic entities (a.k.a. values, objects), and & is a way of relating them. The conversion from one class to another is kind of distracting here, because it really only needs to happen in the special cases like &:sym .

The & is really more like variable assignment ( = ) in ruby rather than an operator, in that '=' affects bindings (another syntax-semantics relation) but does not operate on objects. This is something that often trips up people coming to ruby from C et al.

The only use I can think of for this feature[1] is if you want to conditionally instantiate the Proc, as in:

def defer time
   if time < Time.now
     yield
   else
     pr = Proc.new
     # store pr somewhere and schedule it for execution
   end
end

But, you could just do something like this, right?

def defer(time, &some_block)
   if time < Time.now
     yield # or some_block.call
   else
     pr = some_block
     # store pr somewhere and schedule it for execution
   end
end

Sure, that works, but it always instantiates a Proc (the &some_block does that), even if the yield branch is taken. It's not a big difference, since instantiation and GC are not usually bottlenecks, but it's still something to be aware of.

Cheers,
Russ

P.S. Joel, I seen now why you were inclined toward Rhodes. :slight_smile:

Ruby everywhere :slight_smile:

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

15

Hi --

Russ McBride wrote:

Thanks. Interesting discussion.

given &expr, if expr is a Proc object, use it as
the block; if not, call to_proc on it.

... and then use the resulting proc as a block.

Maybe the best way to think of & is a a proc/block converter, which,
if called on something that is neither looks first for a to_proc method before
performing its conversion.

That still makes & sound like an operator method that can be called on an object to produce a different object, though. Blocks are syntactic entities, Procs are semantic entities (a.k.a. values, objects), and & is a way of relating them. The conversion from one class to another is kind of distracting here, because it really only needs to happen in the special cases like &:sym .

It's not exactly a conversion, though, since as you point out, blocks
aren't objects. In the &expr scenario (I almost typed &expr; as I've
been typing XML all evening :slight_smile: there has to be an actual Proc object
involved. So it's a kind of normalization, rather than conversion, if
that makes sense.

The & is really more like variable assignment ( = ) in ruby rather than an operator, in that '=' affects bindings (another syntax-semantics relation) but does not operate on objects. This is something that often trips up people coming to ruby from C et al.

I think one issue is that there's no good term for the Proc that gets
pressed into service as a block. It's not exactly a block, but just
calling it a Proc doesn't serve to differentiate it from a Proc passed
as a method argument.

The common practice of using &block as the variable that "catches" the
code block is kind of imprecise for the same reason: it's not a block
that's bound to that variable, but a Proc. The term "block" tends to
get spread out to mean more things than the block itself (the
syntactic construct), which I think is kind of too bad but I'm not
sure what the alternative is.

David

···

On Mon, 6 Jul 2009, Joel VanderWerf wrote:

--
David A. Black / Ruby Power and Light, LLC
Ruby/Rails consulting & training: http://www.rubypal.com
Now available: The Well-Grounded Rubyist (http://manning.com/black2\)
Training! Intro to Ruby, with Black & Kastner, September 14-17
(More info: http://rubyurl.com/vmzN\)

16

David A. Black wrote:

I think one issue is that there's no good term for the Proc that gets
pressed into service as a block. It's not exactly a block, but just
calling it a Proc doesn't serve to differentiate it from a Proc passed
as a method argument.

"Eigenproc" ? :stuck_out_tongue:

The common practice of using &block as the variable that "catches" the
code block is kind of imprecise for the same reason: it's not a block
that's bound to that variable, but a Proc. The term "block" tends to
get spread out to mean more things than the block itself (the
syntactic construct), which I think is kind of too bad but I'm not
sure what the alternative is.

def foo(&proc_that_represents_the_callers_block)

... bah, I can't do better.

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407