Method annotation and anonymous functions

ruby-talk
1

I want to start a discussion about two things that Matz talked about at the rubycon. Dave Thomas suggested that we could get Matz' anonymous function by using "def" without a name. Others suggested that if def returned a method object, then annotations could be applied to that object. So here goes my suggestions. As they stand, they are not even parseable by Ruby, but lets start talking about it anyway.

Anonymous functions:

x = def (a=1,b=2) { ... }
y = def(a,b=3) begin
    .....
end

Annotated functions:

class A
    def foo(a, b)
    .visibility :private # how can this be parsed? (the above line has no trailing marker to signal it as one expression
    .returns Integer
    begin
        ....
    end

    def foo2(a, b).visibility(private).returns(Integer) { ... }

    f3 = def foo3(a,b)
    end
       f3.visibility(:private).returns(Integer)

Anonymous, Annotated functions

x = def (a, b).visibility(private).returns(Integer) { ... }
y = def (a, b) { ... }
y.visibility(:private) # apply annotations later
y.returns(:integer)

where for anonation examples:

class Method
    def visibility(v)
       @annotations[:visibility] = v
       return self
    end

    def returns(r)
       @annotations[:returns] = v
       return self
    end
end

Please rake me over the coals because I am so ignorant :wink:

Steve Tuckner

2

I think what most people have in mind is something like this:

  private def foo(a, b, c)
    # ...
  end

  private module_function def bar(a, b, c)
    # ...
  end

Chaining the annotations on multiple lines can be done using '\':

  private \
  module_function \
  doc("This is a docstring") \
  def baz(a, b, c)
    # ...
  end

This also requires changine private() and module_function() to return a
Symbol rather than a Module (i.e. to return their parameter rather than
self).

See RCR#277 for more information.

Paul

···

On Thu, Oct 20, 2005 at 03:26:49AM +0900, stevetuckner wrote:

I want to start a discussion about two things that Matz talked about at
   def foo2(a, b).visibility(private).returns(Integer) { ... }

3

Hi --

I want to start a discussion about two things that Matz talked about at the rubycon. Dave Thomas suggested that we could get Matz' anonymous function by using "def" without a name. Others suggested that if def returned a method object, then annotations could be applied to that object. So here goes my suggestions. As they stand, they are not even parseable by Ruby, but lets start talking about it anyway.

Anonymous functions:

x = def (a=1,b=2) { ... }
y = def(a,b=3) begin
  .....
end

I don't think either of those is exactly what Dave suggested:

   x = def (a=1,b=2)
         ...
       end

without the braces, and without 'begin', just like a method
definition.

I actually like the second iteration of the idea more:

   x = lambda (a=1,b=2)
         ...
       end

using method-definition style for an anonymous function, but a
different keyword. This keeps it more robust; there's a more "solid"
indication that it's something other than a def.

(Actually that was sort of the third iteration :slight_smile: The first was:

   -> (a=1,b=2)
     ...
   end

which I posted here last week in an attempt to at least ameliorate the
arrow thing. But having a real keyword plus the no-braces is
definitely better.)

David

···

On Thu, 20 Oct 2005, stevetuckner wrote:

--
David A. Black
dblack@wobblini.net

4

I just finished (are we ever "finished"?) an annotations system that is
being used by Nitro. Originally I created a class I called a Service
(bad name I know) to manage first class methods so I could annotate
them. It worked okay, but had a lot of overhead. Yet there was one
really bad problem: what about *reusable annotations*? You need to be
able to create annotations w/o necessarily having a method to attach
them to.

I tried a VirtualService but it was even worse. To remedy I decided to
use Symbol instead since we typically use those to reference methods
(perhaps even better would be a sublclass MethodName < Symbol).
Unfortunately without a built in Binding.of_caller it's not very good
either :-(. So I ended up just storing them in a hash in the
class/module they belong to.

Anyway, the point is that annotations are better if they can exist on
their own too --as _potental annotations_, if you will. Something to
consider.

T.

5

Hi,

Anonymous functions:

x = def (a=1,b=2) { ... }
y = def(a,b=3) begin
   .....
end

Probably I will not use "def" for anonymous function, because there
are tow major differences in def statement and anonymous def above:

  * def statement defines new name for a method, anonymous def does
    not define any name.

  * besides that, def statement introduces a whole new scope,
    anonymous def should be a closure, that allows access to external
    local variables.

Annotated functions:

class A
   def foo(a, b)
   .visibility :private # how can this be parsed? (the above
line has no trailing marker to signal it as one expression
   .returns Integer
   begin
       ....
   end

This is a too big syntactic incompatibility.

              matz.

···

In message "Re: Method annotation and anonymous functions" on Thu, 20 Oct 2005 03:26:49 +0900, stevetuckner <stevetuckner@usfamily.net> writes:

6

stevetuckner wrote:

<snip intro>

>

Annotated functions:

class A
   def foo(a, b)
   .visibility :private # how can this be parsed? (the above line has no trailing marker to signal it as one expression
   .returns Integer
   begin
       ....
   end

   def foo2(a, b).visibility(private).returns(Integer) { ... }

   f3 = def foo3(a,b)
   end
     f3.visibility(:private).returns(Integer)

Oh please no! Annotations are, as can be inferred from the name, comments on the
code. Could we just keep these in the comment section? Then we will have a tool
which reads the documentation and can

1) Do RDoc-like external documentation
2) embed documentation in the code itself (in the objects or as separate data)
3) do both 1 and 2 at the same time
4) do nothing if the user wants to disable it.

> <snip rest>

Steve Tuckner

E

7

Paul Brannan wrote:

I want to start a discussion about two things that Matz talked about at def foo2(a, b).visibility(private).returns(Integer) { ... }
   
I think what most people have in mind is something like this:

private def foo(a, b, c)
   # ...
end

private module_function def bar(a, b, c)
   # ...
end

What I don't like about that proposal is that these annotations only take one argument (the symbol) and aren't applied to a Method object (where the annotations rightly should reside).

Chaining the annotations on multiple lines can be done using '\':

private \
module_function \
doc("This is a docstring") \
def baz(a, b, c)
   # ...
end

Aside from looking icky, how does doc() know what symbol to use to apply the annotation with?

···

On Thu, Oct 20, 2005 at 03:26:49AM +0900, stevetuckner wrote:

This also requires changine private() and module_function() to return a
Symbol rather than a Module (i.e. to return their parameter rather than
self).

See RCR#277 for more information.

Paul

8

David A. Black wrote:

Hi --

I want to start a discussion about two things that Matz talked about at the rubycon. Dave Thomas suggested that we could get Matz' anonymous function by using "def" without a name. Others suggested that if def returned a method object, then annotations could be applied to that object. So here goes my suggestions. As they stand, they are not even parseable by Ruby, but lets start talking about it anyway.

Anonymous functions:

x = def (a=1,b=2) { ... }
y = def(a,b=3) begin
  .....
end

I don't think either of those is exactly what Dave suggested:

  x = def (a=1,b=2)
        ...
      end

What I don't like about this is that to do one liners, you need to do:

x = def(a=1, b=2); ...; end

without the braces, and without 'begin', just like a method
definition.

I actually like the second iteration of the idea more:

  x = lambda (a=1,b=2)
        ...
      end

same issue

using method-definition style for an anonymous function, but a
different keyword. This keeps it more robust; there's a more "solid"
indication that it's something other than a def.

I don't mind lambda over def. What object would lambda return? A proc object? Could annotations be applied to it?

···

On Thu, 20 Oct 2005, stevetuckner wrote:

(Actually that was sort of the third iteration :slight_smile: The first was:

  -> (a=1,b=2)
    ...
  end

which I posted here last week in an attempt to at least ameliorate the
arrow thing. But having a real keyword plus the no-braces is
definitely better.)

David

9

Trans wrote:

I just finished (are we ever "finished"?) an annotations system that is
being used by Nitro. Originally I created a class I called a Service
(bad name I know) to manage first class methods so I could annotate
them. It worked okay, but had a lot of overhead. Yet there was one
really bad problem: what about *reusable annotations*? You need to be
able to create annotations w/o necessarily having a method to attach
them to.

I tried a VirtualService but it was even worse. To remedy I decided to
use Symbol instead since we typically use those to reference methods
(perhaps even better would be a sublclass MethodName < Symbol).
Unfortunately without a built in Binding.of_caller it's not very good
either :-(. So I ended up just storing them in a hash in the
class/module they belong to.

Anyway, the point is that annotations are better if they can exist on
their own too --as _potental annotations_, if you will. Something to
consider.

T.

From the replies I got so far, it seems as though "clean solutions" involving method objects are too inefficient. Though I don't know why or even if it is so. Can you post some code showing your annotations addition you did for Nitro?

Steve Tuckner

···
10

I like this too. And there's no reason IMO why we can't have both
anonymous functions and anonymous closures (so def and lambda could have
the same syntax but lambda would create a closure, while def would not).

Then we could also have named closures as well:

  class Foo
    a = 42
    lambda foo(b, c)
      puts a, b, c
    end
  end

The only problem I have with this is that currently lambda is not a
keyword; it is a method that takes a block and returns a Proc object.
Creating new keywords, especially out of existing non-keywords, should
always be done with caution. Currently, the following code works fine:

  def lambda(article)
    puts "Hello from [ruby-talk:#{article}]"
  end

  article = 161640
  lambda(article) #=> Hello from [ruby-talk:161460]

But I would imagine that if lambda were a keyword this would confuse the
parser.

IMO the real issue is that we want to distinguish between blocks that
take arguments by multiple assignment and blocks that take arguments the
"usual" way. One way to do this, then, is to use similar syntax for
both kinds of blocks. For example, consider:

  foo do |a, b|
    # arguments passed by multiple assignment. matz has previously
    # indicated that in the future a and b will also be block-local.
  end

vs:

  foo do <a, b>
    # arguments passed as a function. a and b are block-local
    # regardless of whether they have been previously defined.
  end

  p = proc { <a, b> ... a and b are not passed by MI here ... }

  foo do <&block>
    # an anonymous function that takes a block. you can pass the block
    # from inside foo() with:
    # yield { ... }
  end

This is actually the same syntax that matz proposed in
[ruby-talk:12289] for declaring paramters to be block-local; I'm
extending the idea by suggesting that the angle brackets also change how
the arguments are passed. IIRC there was much controversy over the
<...> syntax at that time, so please don't get too hung up on the syntax
here; consider instead the idea of differentiating between how the
parameters are passed based on some specification of the parameter list
rather than the function itself.

Paul

···

On Thu, Oct 20, 2005 at 04:09:52AM +0900, David A. Black wrote:

I don't think either of those is exactly what Dave suggested:

  x = def (a=1,b=2)
        ...
      end

without the braces, and without 'begin', just like a method
definition.

I actually like the second iteration of the idea more:

  x = lambda (a=1,b=2)
        ...
      end

11

Most languages define some token that specifically states method attribute...
Java uses @... since @ isn't a used char otherwise.

for us ... it would need to be something else ...

and it would simply bind to the function its declared in, just like
functions bind to the classes they are declared in.

how 'bout

def method1( a, b )
  ->visibility = true
  ->returns = int
  # actually do some work...
  ...
end

just thinking out loud... since it would have a new punctuation it
wouldn't introduce incompatibility.

j.

···

On 10/19/05, Yukihiro Matsumoto <matz@ruby-lang.org> wrote:

Hi,

In message "Re: Method annotation and anonymous functions" > on Thu, 20 Oct 2005 03:26:49 +0900, stevetuckner <stevetuckner@usfamily.net> writes:

>Anonymous functions:
>
>x = def (a=1,b=2) { ... }
>y = def(a,b=3) begin
> .....
>end

Probably I will not use "def" for anonymous function, because there
are tow major differences in def statement and anonymous def above:

  * def statement defines new name for a method, anonymous def does
    not define any name.

  * besides that, def statement introduces a whole new scope,
    anonymous def should be a closure, that allows access to external
    local variables.

>Annotated functions:
>
>class A
> def foo(a, b)
> .visibility :private # how can this be parsed? (the above
>line has no trailing marker to signal it as one expression
> .returns Integer
> begin
> ....
> end

This is a too big syntactic incompatibility.

                                                        matz.

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

12

Trans wrote:

To remedy I decided to
use Symbol instead since we typically use those to reference methods
(perhaps even better would be a sublclass MethodName < Symbol).

That would be the best solution. Specially if MethodName was aware of
namespaces, giving us a hook for selector namespaces!

13

A couple thoughts.

For annotations to be manipulated via tools there
has to be a stable/standard syntax. If the annotations appear
as Ruby comments then you'll have to invent some other parser to extract
information from the comments and the information won't be available at
runtime (no meta-programming based on annotations).

If the annotations are part of the Ruby syntax then the Ruby parser
and meta-programming tools can come into play.

In order to be useful across classes/gems/packages/authors there needs to
be some standard semantics associated with the annotations. Otherwise
different authors will choose different meanings for the annotation names/identifiers.
This is true for annotations done in comments or as part of an extended
Ruby syntax.

Gary Wright

···

On Oct 20, 2005, at 12:14 AM, ES wrote:

Oh please no! Annotations are, as can be inferred from the name, comments on the
code. Could we just keep these in the comment section? Then we will have a tool
which reads the documentation and can

1) Do RDoc-like external documentation
2) embed documentation in the code itself (in the objects or as separate data)
3) do both 1 and 2 at the same time
4) do nothing if the user wants to disable it.

14

What I don't like about that proposal is that these annotations only
take one argument (the symbol) and aren't applied to a Method object
(where the annotations rightly should reside).

Changing def to return a Method object could also be done, but this is a
larger change; it requires methods like private() to be able to take a
Method or a Symbol (currently they only take a Symbol).

The annotation does not belong with the (Unbound)Method object; it
belongs with the method. This is an important distinction:

  irb(main):001:0> class Foo; def foo; end; end
  => nil
  irb(main):005:0> m1 = Foo.instance_method(:foo); m1.object_id
  => 538483812
  irb(main):006:0> m2 = Foo.instance_method(:foo); m2.object_id
  => 538477272

Note that m1 and m2 are two *different* objects which refer to the same
method.

>Chaining the annotations on multiple lines can be done using '\':
>
> private \
> module_function \
> doc("This is a docstring") \
> def baz(a, b, c)
> # ...
> end
>
>
Aside from looking icky, how does doc() know what symbol to use to apply
the annotation with?

Good point, I missed this the first time through.

Paul

···

On Thu, Oct 20, 2005 at 04:41:51AM +0900, stevetuckner wrote:

15

I like this too. And there's no reason IMO why we can't have both
anonymous functions and anonymous closures (so def and lambda could have
the same syntax but lambda would create a closure, while def would not).

I agree. See the thread "Anonymous methods, blocks etc. (Cont.
'default block params')" starting at ruby-talk/160709 for some recent
discussion about this.

Then we could also have named closures as well:

class Foo
   a = 42
   lambda foo(b, c)
     puts a, b, c
   end
end

That's an interesting idea. It would certainly be orthogonal. However,
you can already do that with define_method:

  class Foo
    a = 42
    define_method(:foo) do |b, c|
      puts a, b, c
    end
  end

  Foo.new.foo(1,2)
  __END__
  42
  1
  2

The only problem I have with this is that currently lambda is not a
keyword; it is a method that takes a block and returns a Proc object.
Creating new keywords, especially out of existing non-keywords, should
always be done with caution.

Agreed.

Currently, the following code works fine:

[snip]

But I would imagine that if lambda were a keyword this would confuse the
parser.

Well, I doubt it would break ~much~ code. It would be quite an unusual
thing to define a function named lamdba(), no?

[snip discussion about || vs <> block local syntax]

I think you're on a losing wicket here. It seems that the zeitgeist is
with the semi-colon to introduce block locals.

Regards,

Sean

···

On 10/19/05, Paul Brannan <pbrannan@atdesk.com> wrote:

16

Of course, my use of '->' doesn't take into consideration the
suggestions for new style syntax for blocks ->( a, b ) { ... }

...

You've always done a great job with things, I guess I'm just
suggesting another bit of punctuation that would state ( attribute of
surrounding method ).

j.

···

On 10/19/05, Jeff Wood <jeff.darklight@gmail.com> wrote:

Most languages define some token that specifically states method attribute...
Java uses @... since @ isn't a used char otherwise.

for us ... it would need to be something else ...

and it would simply bind to the function its declared in, just like
functions bind to the classes they are declared in.

how 'bout

def method1( a, b )
  ->visibility = true
  ->returns = int
  # actually do some work...
  ...
end

just thinking out loud... since it would have a new punctuation it
wouldn't introduce incompatibility.

j.

On 10/19/05, Yukihiro Matsumoto <matz@ruby-lang.org> wrote:
> Hi,
>
> In message "Re: Method annotation and anonymous functions" > > on Thu, 20 Oct 2005 03:26:49 +0900, stevetuckner <stevetuckner@usfamily.net> writes:
>
> >Anonymous functions:
> >
> >x = def (a=1,b=2) { ... }
> >y = def(a,b=3) begin
> > .....
> >end
>
> Probably I will not use "def" for anonymous function, because there
> are tow major differences in def statement and anonymous def above:
>
> * def statement defines new name for a method, anonymous def does
> not define any name.
>
> * besides that, def statement introduces a whole new scope,
> anonymous def should be a closure, that allows access to external
> local variables.
>
> >Annotated functions:
> >
> >class A
> > def foo(a, b)
> > .visibility :private # how can this be parsed? (the above
> >line has no trailing marker to signal it as one expression
> > .returns Integer
> > begin
> > ....
> > end
>
> This is a too big syntactic incompatibility.
>
> matz.
>
>

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

17

Hi --

···

On Thu, 20 Oct 2005, Jeff Wood wrote:

Most languages define some token that specifically states method attribute...
Java uses @... since @ isn't a used char otherwise.

for us ... it would need to be something else ...

and it would simply bind to the function its declared in, just like
functions bind to the classes they are declared in.

how 'bout

def method1( a, b )
->visibility = true
->returns = int
# actually do some work...
...
end

just thinking out loud... since it would have a new punctuation it
wouldn't introduce incompatibility.

Worse: it would introduce new punctuation :slight_smile:

David

--
David A. Black
dblack@wobblini.net

18

Whoa! You may have just killed many a big bird with one ity bity hook!
You give me notions, but could you elborate more?

T.

···

itsme...@hotmail.com wrote:

Trans wrote:
> To remedy I decided to
> use Symbol instead since we typically use those to reference methods
> (perhaps even better would be a sublclass MethodName < Symbol).

That would be the best solution. Specially if MethodName was aware of
namespaces, giving us a hook for selector namespaces!

19

I realize now that I should have written:

  private \
  module_function \
  doc "This is a docstring", \
  def baz(a, b, c)
  end

You are right that this is ugly (and it produces warnings). But it
avoids introducting a new syntax.

It's also possible to use Binding.of_caller and method_added to do
method annotations. Use of such a library would look like this:

  private
  module_function
  doc "This is a docstring"
  def baz(a, b, c)
  end

Paul

···

On Thu, Oct 20, 2005 at 04:41:51AM +0900, stevetuckner wrote:

> private \
> module_function \
> doc("This is a docstring") \
> def baz(a, b, c)
> # ...
> end
>
>
Aside from looking icky, how does doc() know what symbol to use to apply
the annotation with?

20

Yes it would, but I don't see another way to do attribution of methods
without causing compatibility issues .... do you?

j.

···

On 10/19/05, David A. Black <dblack@wobblini.net> wrote:

Hi --

On Thu, 20 Oct 2005, Jeff Wood wrote:

> Most languages define some token that specifically states method attribute...
> Java uses @... since @ isn't a used char otherwise.
>
> for us ... it would need to be something else ...
>
> and it would simply bind to the function its declared in, just like
> functions bind to the classes they are declared in.
>
> how 'bout
>
> def method1( a, b )
> ->visibility = true
> ->returns = int
> # actually do some work...
> ...
> end
>
> just thinking out loud... since it would have a new punctuation it
> wouldn't introduce incompatibility.

Worse: it would introduce new punctuation :slight_smile:

David

--
David A. Black
dblack@wobblini.net

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood