Lisp macros

Judge for yourself:

  http://ll2.ai.mit.edu/

···

On 9/27/05, Joe Van Dyk <joevandyk@gmail.com> wrote:

On 9/26/05, Jim Freeze <jim@freeze.org> wrote:
> On 9/26/05, James Britt <james_b@neurogami.com> wrote:
> > Joe Van Dyk wrote:
> >
> Matz has said (at the Lightweight Language Conf) that Ruby will never
> have macros because they are too easily abused, by the average person,
> to mutate the language.

Was he joking?

--
Jim Freeze

Alright, now I am intrigued. How much work would it be to create a
Ruby Lisp-like parser?

class Risp
  def Risp.eval(code) # code should be an array!
    # What happens here? We need to create a string to be eval'ed. As
    # the string is created, if we find an array jump in to eval that first.
    # -- We need a macro type that would change the format of what is being
    # parsed.
    # -- We'd need to decide how to handle objects since methods are not
    # members in Lisp. Maybe make the first parameter always be the self so
    # [x, y, z] becomes y.x(z)

···

On 9/27/05, Ben <benbelly@gmail.com> wrote:

(much rambling snipped)

    #
    # What a totally pointless, but entertaining exercise. :slight_smile:
    # I don't think it could go anywhere. :frowning:
  end
end

Hmm, that is strong language. Do you think Matz has made a bad decision
here? Ruby does have lambda's. If you really need a lisp like language,
there is always Lisp. :slight_smile:

Seriously, can you put your statement in words that define exactly
what you mean? What exacly 'macro like' feature are you looking for?
Are you really looking for macro capability or just a lazy eval?

···

On 9/27/05, Ben <benbelly@gmail.com> wrote:

On 9/27/05, Jim Freeze <jim@freeze.org> wrote:
>
> Matz has said (at the Lightweight Language Conf) that Ruby will never
> have macros because they are too easily abused, by the average person,
> to mutate the language.
>

  This is a cop-out. Every time a new language comes around and

--
Jim Freeze

Robbie Carlton wrote:

code as data is what makes lisp macros so powerful.

In a c macro you have the power of a little macro language to
manipulate strings, in a lisp macro you have the power of lisp
language to manipulate lisp data structures. I'll elaborate.

When the compiler sees a call to a macro it passes all of the
arguments to the macro, before evaluating.
so in a call like

(dotimes (i 10)
(do-some-crazy-thing)
(print i))

the macro dotimes is passed the list (i 10) and the list
((do-some-crazy-thing) (print i))
the macro can then operate on them as data, using the full power of
lisp. At the end of all this, the macro returns another piece of
data, a list, which is then interpreted as a lisp form, code, and
evaluated.

If lisp code wasn't the same as lisp data, macros would be a lot less
powerful as you would have to have two languages, one for
manipulating lisp data, and another for manipulating lisp code (this
is what you get in c).

Hope this helps. Sorry the example isn't very enlightening, it's
difficult to illustrate the power of macros in (< 500 words) as
they're very complicated. If you'd like to know more* ... learn lisp
:slight_smile:

I'm by no means a Lisp expert (rather beginner) but this matches my
understanding of Lisp macros pretty well. Maybe you can add to that that
a Lisp macro invocation looks exactly like a function invocation which at
the same time integrates it seamlessly into the language and makes it
harder to read / learn Lisp.

Maybe I missed something in the discussion but I still wonder what would
be gained by having Lisp macro like capabilities in Ruby (apart from the
fun that it'd certainly be for some of us)? Do we actually gain
something? Is there something that we cannot do in Ruby without this
feature and that will become utterly easy with it? Don't get me wrong, I
don't want to put this down - I'm just wondering about the benefits.

Kind regards

    robert

+1

As Paul Graham points out, the point of macros is syntactic
abstraction. If you find yourself writing similar code over and over
or find that you're using a pattern over and over, you write out a
macro and abstract away that similarity. This makes code easier to
understand and increases maintainability by reducing multiple
instances of the same concept into a single macro.

The difference is analagous to that between procedural and
object-oriented programming. It is difficult to explain to a
procedural programmer why objects are so useful ("But what use are
classes? I can just use different functions and name them
appropriately."). The case is the same with macros; they are a
different way of thinking, and provide additional power that non-Lisp
languages do not offer.

Ruby's blocks can get you a lot of what Lisp macros can, but they are
not a complete substitute. For instance, the dotimes macro Robbie
demonstrated is possible in Ruby:

The dotimes macro could be implemented (and is implemented) as

  10.times do |i|
    ..
  end

The two bits of code are functionally identical, but it's clear that
the Ruby version is jumping through more hoops than the macro version
to present a counting abstraction to the user

(dotimes (i 10)
    ....)

The Ruby version must contort itself to work with Ruby's block syntax,
while the macro version has no such restrictions to deal with. I am
not digging on Ruby, but just trying to point out why macros are
different. The dotimes macro can choose when and how its parameters
are evaluated. So i is not treated as a variable name simply because
dotimes does not want to treat it that way. The code passed as "..."
is not run directly, but is instead run in the way dotimes wants it to
(that is, it is executed 10 times).

Hopefully, someone can supply some really cool examples. I haven't
done enough Lisp to have any, but Practical Common Lisp is a
good place to start reading about cool macros.

Bill

···

On 9/27/05, Robbie Carlton <robbie.carlton@gmail.com> wrote:

code as data is what makes lisp macros so powerful.

In a c macro you have the power of a little macro language to manipulate
strings, in a lisp macro you have the power of lisp language to manipulate
lisp data structures. I'll elaborate.

When the compiler sees a call to a macro it passes all of the arguments to
the macro, before evaluating.
so in a call like

(dotimes (i 10)
(do-some-crazy-thing)
(print i))

the macro dotimes is passed the list (i 10) and the list
((do-some-crazy-thing) (print i))
the macro can then operate on them as data, using the full power of lisp. At
the end of all this, the macro returns another piece of data, a list, which
is then interpreted as a lisp form, code, and evaluated.

If lisp code wasn't the same as lisp data, macros would be a lot less
powerful as you would have to have two languages, one for manipulating lisp
data, and another for manipulating lisp code (this is what you get in c).

Hope this helps. Sorry the example isn't very enlightening, it's difficult
to illustrate the power of macros in (< 500 words) as they're very
complicated. If you'd like to know more* ... learn lisp :slight_smile:

Now I will stop with the lisp advocacy.

*on lisp by Paul Graham is the bible on macros, but it's a little hard for
someone who's new to lisp, try practical common lisp by Peter Siebel, (both
are available free online).

On 9/27/05, Devin Mullins <twifkak@comcast.net> wrote:
>
> Devin Mullins wrote:
>
> > The big thing Lisp has that Ruby can't do is code-as-data. I wish I
> > could provide a good example of how that might be used practically,
> > but it's been quite a while since I touched lisp.
> >
> > Devin
>
> Well, nobody's provided an example. The best I can do is provide more
> explanation. Since code is just (blah blah (blah blah)) -- that is, just
> a list -- and Lisp has the capability to delay evaluation (I think with
> an '), you can pass code into another function and have that function
> actually manipulate the internals of your code, and build some new code
> out of it. You could, maybe, write something that stripped all the print
> statements out at runtime... Crappy example...
>
> Help! Is there a Lisper in the house?
>
> Devin
>
>
>

--
Bill Atkins

Edward Faulkner ha scritto:

Crappy example...

Help! Is there a Lisper in the house?

Part of the attraction of Lisp macros is that they allow you to keep
the core language very small, and add the features you want with
macros. For example, as long as you have the lambda operator, you can
define things like "while" and "let" as macros, not keywords. This is
in fact how many Lisp implementations work, because it keeps the
compiler/interpreter simple.

I may recall wrongly, but my I remember that
- once you have lambda forms and macros you can write everything
- Common Lisp, which is what I think people think of when writing "lisp", still has something like X special forms (with X>20)

<snip>

Paul Graham's "On Lisp" is available online and contains several
chapters about macros. It gives many more examples:

I think another nice text available online for lisp noobs (like me) is "Practical Common Lisp". It is not a whole macro cookbook like Graham's text, but it shows how macros are really a simple concept that can be applied to simple abstractions, that stacked on over another result in much smaller and cleaner code

···

On Tue, Sep 27, 2005 at 09:21:07PM +0900, Devin Mullins wrote:

I'm not meaning to be insulting, but saying that something is too
dangerous to put in the hands of the uneducated is almost always a way
of evading a question rather than answering it. How about:
"The goals I am pursuing with Ruby (list goals) will not be achieved
any better with macros"
"The syntax and language structure I like for Ruby will not support
macros in an intuitive, maintainable way" (I certainly agree with
this)
"I don't believe the benefits of macros are worth the effort of
supporting them in the interpreter and training developers to use
them."

If the reason is really that they are too scary, say why:
"The delayed evaluation necessary to support macros runs contrary to
the rest of the language. This lack of clarity will prove a hindrance
to the quick development I intended Ruby to support. Also, exporting
to the client of a class those implementation details (method or
macro) violates encapsulation which will increase the learning curve
of the language."

I'm probably overreacting, but I feel patronized when I'm told I can't
have something because it's "too easy to abuse". :slight_smile:

-Ben

···

On 9/27/05, Jim Freeze <jim@freeze.org> wrote:

On 9/27/05, Ben <benbelly@gmail.com> wrote:
>
> This is a cop-out. Every time a new language comes around and

Hmm, that is strong language. Do you think Matz has made a bad decision
here? Ruby does have lambda's. If you really need a lisp like language,
there is always Lisp. :slight_smile:

You might be interested in Rouge, a (simple) Lisp interpreter in Ruby:

http://raa.ruby-lang.org/project/rouge/

Sean

···

On 9/27/05, Ben <benbelly@gmail.com> wrote:

On 9/27/05, Ben <benbelly@gmail.com> wrote:
> (much rambling snipped)

Alright, now I am intrigued. How much work would it be to create a
Ruby Lisp-like parser?

Matz has said (at the Lightweight Language Conf) that Ruby will never
have macros because they are too easily abused, by the average person,
to mutate the language.

  This is a cop-out. Every time a new language comes around and

Hmm, that is strong language. Do you think Matz has made a bad decision
here? Ruby does have lambda's. If you really need a lisp like language,
there is always Lisp. :slight_smile:

I think I agree with Ben to some degree. I don't think the outcome is a bad one, but the reason seems like a 'cop-out'. Surely there's a better reason than that. Even a "I don't want to" or "I don't see why" or "Ruby takes a different approach" or "No" from Matz would be much better. Saying that it is "too dangerous" is just asking for an argument :slight_smile:

Seriously, can you put your statement in words that define exactly
what you mean? What exacly 'macro like' feature are you looking for?
Are you really looking for macro capability or just a lazy eval?

Looking at macros on a feature by feature basis isn't going to work. Just like learning about macros by reading about what you can do with them isn't going to work either -- at least it didn't for me. Having macros like Common Lisp has changes the way you think about programming and how you do it -- they certainly changes the programmes. I really like working in both Common Lisp and Ruby. Ruby is very very flexible and probably can address most of the individual features/aspects of macros, but it cannot match macros as a concept because it simply doesn't have the concept. And that is okay, let Ruby be Ruby.

Foolishly venturing into very dangerous territory to actually address the question... Ruby seems to be based on a few syntactic structures and a whole bunch of method calls. Anybody can add methods to be called, so extending Ruby's 'syntax' that way is easy. Adding new funny syntactic structures is what macros are for. You can't do that in Ruby. You can in lisp. So, there would be one use for macros that I don't think Ruby can deal with right now.

Cheers,
Bob

···

On Sep 27, 2005, at 10:09 AM, Jim Freeze wrote:

On 9/27/05, Ben <benbelly@gmail.com> wrote:

On 9/27/05, Jim Freeze <jim@freeze.org> wrote:

--
Jim Freeze

----
Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/&gt;
Recursive Design Inc. -- <http://www.recursive.ca/&gt;
Raconteur -- <http://www.raconteur.info/&gt;

Lisp macros are essentially a technique for metaprogramming. Ruby
already has its own metaprogramming techniques. I suspect almost
anything you can do with one you can do with the other.

But they take very different approaches. Lisp enables metaprogramming
by making the full power of the language available at compile time.
Ruby enables metaprogramming by eliminating the distinction between
compile time and run time.

(Note that Lisp can be run interpreted too. Compilation in that case
happens just before each evaluation.)

Personally, I think the Lisp way is more elegant, and it has more
opportunities for optimization. But I think it would be a mistake to
try to graft it onto Ruby. It doesn't match with Ruby's style and
philosophy.

regards,
Ed

···

On Wed, Sep 28, 2005 at 12:26:42AM +0900, Robert Klemme wrote:

Maybe I missed something in the discussion but I still wonder what would
be gained by having Lisp macro like capabilities in Ruby (apart from the
fun that it'd certainly be for some of us)? Do we actually gain
something? Is there something that we cannot do in Ruby without this
feature and that will become utterly easy with it?

Coincidentally, in the thread "Is there an issue with a library extending Object" I've just described another place where lisp macros would come in handy. The thread isn't very long so it is probably best to read about it there.

Cheers,
Bob

···

On Sep 27, 2005, at 10:09 AM, Jim Freeze wrote:

What exacly 'macro like' feature are you looking for?

----
Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/&gt;
Recursive Design Inc. -- <http://www.recursive.ca/&gt;
Raconteur -- <http://www.raconteur.info/&gt;

Based on the discussion, if he's watching ( Matz? ) ... would you please
enlighten us with your feelings?

... I for one have never minded having "too much power". It's the exercise
of responsibility that makes programming an art.

j.

···

On 9/27/05, Ben <benbelly@gmail.com> wrote:

On 9/27/05, Jim Freeze <jim@freeze.org> wrote:
> On 9/27/05, Ben <benbelly@gmail.com> wrote:
> >
> > This is a cop-out. Every time a new language comes around and
>
> Hmm, that is strong language. Do you think Matz has made a bad decision
> here? Ruby does have lambda's. If you really need a lisp like language,
> there is always Lisp. :slight_smile:
>

I'm not meaning to be insulting, but saying that something is too
dangerous to put in the hands of the uneducated is almost always a way
of evading a question rather than answering it. How about:
"The goals I am pursuing with Ruby (list goals) will not be achieved
any better with macros"
"The syntax and language structure I like for Ruby will not support
macros in an intuitive, maintainable way" (I certainly agree with
this)
"I don't believe the benefits of macros are worth the effort of
supporting them in the interpreter and training developers to use
them."

If the reason is really that they are too scary, say why:
"The delayed evaluation necessary to support macros runs contrary to
the rest of the language. This lack of clarity will prove a hindrance
to the quick development I intended Ruby to support. Also, exporting
to the client of a class those implementation details (method or
macro) violates encapsulation which will increase the learning curve
of the language."

I'm probably overreacting, but I feel patronized when I'm told I can't
have something because it's "too easy to abuse". :slight_smile:

-Ben

--
"http://ruby-lang.org -- do you ruby?"

Jeff Wood

Well, it goes without saying that I am not Matz and that I am only giving
my version of what Matz said at the LLC. I encourage you to watch
the video and give us your opinion.

   http://ll2.ai.mit.edu/

IIRC, he is in the afternoon session near the front.

···

On 9/27/05, Ben <benbelly@gmail.com> wrote:

On 9/27/05, Jim Freeze <jim@freeze.org> wrote:
> On 9/27/05, Ben <benbelly@gmail.com> wrote:

I'm probably overreacting, but I feel patronized when I'm told I can't
have something because it's "too easy to abuse". :slight_smile:

--
Jim Freeze

---<other good stuff (maybe better) snipped>----

I'm probably overreacting, but I feel patronized when I'm told I can't
have something because it's "too easy to abuse". :slight_smile:

+1

Randy Kramer

···

On Tuesday 27 September 2005 10:42 am, Ben wrote:

Having
macros like Common Lisp has changes the way you think about
programming and how you do it

   Absolutely. Well said.

because it simply doesn't have the concept. And that is okay, let
Ruby be Ruby.

  I agree wholeheartedly. I am really enjoying learning ruby, and I
am quite happy to have it instead of Lisp for many things. Just the
fact that I don't have to choose some third party tool to use the
network or regular expressions is awesome.

-Ben

···

On 9/27/05, Bob Hutchison <hutch@recursive.ca> wrote:

Bob Hutchison schrieb:

Ruby seems to be based on a few syntactic structures and a whole bunch of method calls. Anybody can add methods to be called, so extending Ruby's 'syntax' that way is easy. Adding new funny syntactic structures is what macros are for.

And as I understand it, this is exactly what Matz didn't want to happen.

You can't do that in Ruby. You can in lisp. So, there would be one use for macros that I don't think Ruby can deal with right now.

I also think this is the main difference, though I haven't been programming in Lisp since more than 13 years.

Regards,
Pit

> Maybe I missed something in the discussion but I still wonder what would
> be gained by having Lisp macro like capabilities in Ruby (apart from the
> fun that it'd certainly be for some of us)? Do we actually gain
> something? Is there something that we cannot do in Ruby without this
> feature and that will become utterly easy with it?

Lisp macros are essentially a technique for metaprogramming. Ruby
already has its own metaprogramming techniques. I suspect almost
anything you can do with one you can do with the other.

But they take very different approaches. Lisp enables metaprogramming
by making the full power of the language available at compile time.
Ruby enables metaprogramming by eliminating the distinction between
compile time and run time.

(Note that Lisp can be run interpreted too. Compilation in that case
happens just before each evaluation.)

I believe early versions were actually interpreted only and you definitely can have a Lisp interpreter that does no compilation (you wouldn't want it for speed reasons but it's perfectly feasible). I still don't view Lisp as a compiled language. As far as I understand the language the runtime must contain a compiler if you want compiled code. To me this is a definite hint that compilation is rather an optimization technique than a feature of the language (as opposed to C etc.). Am I wrong here?

Personally, I think the Lisp way is more elegant, and it has more
opportunities for optimization. But I think it would be a mistake to
try to graft it onto Ruby. It doesn't match with Ruby's style and
philosophy.

That pretty much covers what I suspected.

Kind regards

    robert

···

Edward Faulkner <ef@alum.mit.edu> wrote:

On Wed, Sep 28, 2005 at 12:26:42AM +0900, Robert Klemme wrote:

Was that +1 to overreacting or to feeling patronized? :slight_smile:
Sorry, couldn't resist.

···

On 9/27/05, Randy Kramer <rhkramer@gmail.com> wrote:

On Tuesday 27 September 2005 10:42 am, Ben wrote:
---<other good stuff (maybe better) snipped>----

> I'm probably overreacting, but I feel patronized when I'm told I can't
> have something because it's "too easy to abuse". :slight_smile:

+1

--
Jim Freeze

All of the major implementations, with the exception of CLISP, are native compilers. I know many, if not all, no longer have interpreters at all. The only time you ever need to compile at runtime is if you call eval, load an uncompiled source file, or change some code on the fly (e.g. in the debugger or shell).

Cheers,
bob

···

On Sep 27, 2005, at 4:26 PM, Robert Klemme wrote:

I believe early versions were actually interpreted only and you definitely can have a Lisp interpreter that does no compilation (you wouldn't want it for speed reasons but it's perfectly feasible). I still don't view Lisp as a compiled language. As far as I understand the language the runtime must contain a compiler if you want compiled code. To me this is a definite hint that compilation is rather an optimization technique than a feature of the language (as opposed to C etc.). Am I wrong here?

----
Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/&gt;
Recursive Design Inc. -- <http://www.recursive.ca/&gt;
Raconteur -- <http://www.raconteur.info/&gt;

Jim Freeze ha scritto:

···

On 9/27/05, Ben <benbelly@gmail.com> wrote:

On 9/27/05, Jim Freeze <jim@freeze.org> wrote:

On 9/27/05, Ben <benbelly@gmail.com> wrote:

I'm probably overreacting, but I feel patronized when I'm told I can't
have something because it's "too easy to abuse". :slight_smile:

Well, it goes without saying that I am not Matz and that I am only giving
my version of what Matz said at the LLC. I encourage you to watch
the video and give us your opinion.

   http://ll2.ai.mit.edu/

IIRC, he is in the afternoon session near the front.

maybe this thread can be interesting
http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-talk/108143?108055-108481+split-mode-vertical