How are Ruby iterators implemented?

How are iterators implemented in Ruby?

Sometime back I had asked a questuin about invoking iterators in
locksteps and I was told that iterators in ruby are implemented as
closures that are passed to the called method. If iterators are passed
as closures then,
    1) Every consumer of the iterator should undergo closure conversion
in some way
    2) How is "break" implemented ?
This similar to how it would be done in Scheme or similar language that
inherently supports closures.

I was reading about CLU (the parent language for the iterator concept),
and they implemnted iterators on a single stack. I believe that that
would involve have caller and callee function frames to live on the
stack at the same time. The iteration would be implemnetd as goto
between these functions with a corresposding shift in stock location to
refernce locals. This implementation has the inherent limitation of not
being able to do multiple itetarors in lock step.

Another alternative is that the itetror is converted to an object that
maintains state - a classical generator. This is what C# and Py do.

How does Ruby do iterators ?

Roshan

What I'm getting ready to tell you is as far as I understand it; it
may not be completely correct -- and the most authoritative way to
see this is the source.

That said, simple iterators like #each implement a loop through the
container and yield the object in the iterator to the provided code
block. #each suspends execution until the block (closure) finishes
executing. That would generally indicate a change in stack frame.

Ruby does have generators available, but they're overkill for ~95%
of cases.

-austin

···

On 4/25/05, Roshan James <roshanj@microsoft.com> wrote:

How are iterators implemented in Ruby?

Sometime back I had asked a questuin about invoking iterators in
locksteps and I was told that iterators in ruby are implemented as
closures that are passed to the called method. If iterators are
passed as closures then, 1) Every consumer of the iterator should
undergo closure conversion in some way 2) How is "break"
implemented ? This similar to how it would be done in Scheme or
similar language that inherently supports closures.

I was reading about CLU (the parent language for the iterator
concept), and they implemnted iterators on a single stack. I
believe that that would involve have caller and callee function
frames to live on the stack at the same time. The iteration would
be implemnetd as goto between these functions with a corresposding
shift in stock location to refernce locals. This implementation
has the inherent limitation of not being able to do multiple
itetarors in lock step.

Another alternative is that the itetror is converted to an object
that maintains state - a classical generator. This is what C# and
Py do.

How does Ruby do iterators ?

--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca

"Austin Ziegler" <halostatue@gmail.com> schrieb im Newsbeitrag
news:9e7db9110504250552129835ec@mail.gmail.com...

> How are iterators implemented in Ruby?
>
> Sometime back I had asked a questuin about invoking iterators in
> locksteps and I was told that iterators in ruby are implemented as
> closures that are passed to the called method.

The iterator is not the closure. The closure (aka block) is just for
executing some piece of code on each element it's given. The iteration
itself is typically implemented in #each (or any other method). Most of
the time (i.e. if you do not use Generator) you cannot access an iterator.
You just have a method that accepts a block.

You can easily implement an iteration yourself, like

class Demo
  def initialize(st,en) @start, @end = st, en end

  def each
    i = @start

    while i <= @end
      yield i
      i += 1
    end

    self
  end
end

Demo.new(1,5).each {|x| puts x}

1
2
3
4
5
=> #<Demo:0x1018f118 @end=5, @start=1>

If iterators are
> passed as closures then, 1) Every consumer of the iterator should
> undergo closure conversion in some way

I'm not sure what you're getting at here. Since there is no iterator I'd
say that in itself is reason enough that there is no closure conversion.
:slight_smile: Other than that, I don't see why a closure should be converted.
Maybe I'm lacking understanding of other languages.

> 2) How is "break"
> implemented ? This similar to how it would be done in Scheme or
> similar language that inherently supports closures.

Does this illustrate the behavior?

Demo.new(1,5).each {|x| puts x; break "end" if x == 4}

1
2
3
4
=> "end"

"break" simply terminates the iterating method (#each in this case) and
forces return of the provided value (if there is any, nil is the default).

> I was reading about CLU (the parent language for the iterator
> concept), and they implemnted iterators on a single stack. I
> believe that that would involve have caller and callee function
> frames to live on the stack at the same time. The iteration would
> be implemnetd as goto between these functions with a corresposding
> shift in stock location to refernce locals. This implementation
> has the inherent limitation of not being able to do multiple
> itetarors in lock step.

No, we actually have a method call here. That's quite obvious if you
choose the other alternative for the implementation:

class Demo
  def each2(&b)
    i = @start

    while i <= @end
      b.call( i )
      i += 1
    end

    self
  end
end

Demo.new(1,5).each2 {|x| puts x}

1
2
3
4
5
=> #<Demo:0x1016cd20 @end=5, @start=1>

> Another alternative is that the itetror is converted to an object
> that maintains state - a classical generator. This is what C# and
> Py do.
>
> How does Ruby do iterators ?

What I'm getting ready to tell you is as far as I understand it; it
may not be completely correct -- and the most authoritative way to
see this is the source.

That said, simple iterators like #each implement a loop through the
container and yield the object in the iterator to the provided code
block. #each suspends execution until the block (closure) finishes
executing. That would generally indicate a change in stack frame.

Yes, you can view a block as an anonymous callback function.

Ruby does have generators available, but they're overkill for ~95%
of cases.

Perfectly agree.

Kind regards

    robert

···

On 4/25/05, Roshan James <roshanj@microsoft.com> wrote:

Austin Ziegler <halostatue@gmail.com> writes:

Ruby does have generators available, but they're overkill for ~95%
of cases.

And too slow in the 5% where you need them? }}:slight_smile:

···

-austin

--
Christian Neukirchen <chneukirchen@gmail.com> http://chneukirchen.org

I wouldn't know. I've not yet needed them :wink:

-austin

···

On 4/25/05, Christian Neukirchen <chneukirchen@gmail.com> wrote:

Austin Ziegler <halostatue@gmail.com > writes:
> Ruby does have generators available, but they're overkill for ~95%
> of cases.
And too slow in the 5% where you need them? }}:slight_smile:

--
Austin Ziegler * halostatue@gmail.com
               * Alternate: austin@halostatue.ca