Is iterating in lock-step possible?

ruby-talk
1

Thanks for all the mails. Like William called out, this did not solve
the underlying problem.

<Repeat>
In my post, I chose the array.each only as an example. Its not arrays
for which I want to do this with - I want to solve the problem for
iterator calls. If zip() internally creates a list of values from both
iterations, then it does not help me. I want to be able to do the actual
computation of the iterator calls in lock-step.
</Repeat>

I was trying to write code where I can have two or more computations
which generate values that I need to compare - with the ruby syntax
tying iterator usage to the code blocks by syntax, there is no way I can
do this. This is especially applicable when the iterators potentially
return an infinite stream of values. (Except for using contiuations...
Which is not really so much of a solution, because I might as well not
have used an iterator in a first place if I could create good
coroutines)

Yes I understand that I take a hit (wrt perf by choosing ruby), but
that's a BAD argument to favour wrapping state first by a iterators and
then once over by a continuation.

On a related note, I had asked this question about C# and it looks like
C# can do both of these cleanly -
foreach(<type> t in foo())
{
  foreach(<type> t in bar())
  {
    //nested
  }
}

And -

IEnumerator<int> ie1 = a.foo().GetEnumerator();
IEnumerator<int> ie2 = a.bar().GetEnumerator();

while (ie1.MoveNext() && ie2.MoveNext())
{
  //lockstep
}

Since Ruby is open to (I believe) a major language redesign these days,
this might be a good feature to consider adding.

Roshan

2

Excerpts from Roshan James's mail of 11 Mar 2005 (EST):

On a related note, I had asked this question about C# and it looks like
C# can do both of these cleanly -
foreach(<type> t in foo())
{
  foreach(<type> t in bar())
  {
    //nested
  }
}

I think you might be overlooking the obvious, which is that you can take
this approach in Ruby. You're writing the objects that do the
computations, right? Simply give them #next? and #next methods and use a
while loop. That's what you'd be doing in these other languages.

The prevalence of internal iterators in Ruby's API is simply a
reflection of the fact that they're so nice to use. It's not the only
way to iterate.

···

--
William <wmorgan-ruby-talk@masanjin.net>

3

I was trying to write code where I can have two or more computations
which generate values that I need to compare - with the ruby syntax
tying iterator usage to the code blocks by syntax, there is no way I can
do this. This is especially applicable when the iterators potentially
return an infinite stream of values. (Except for using contiuations...
Which is not really so much of a solution, because I might as well not
have used an iterator in a first place if I could create good
coroutines)

Continuations are the right answer in this case.

module Enumerable
  attr_accessor :co, :st, :res
  
  def nextco
    each { |e|
      callcc { |cc|
        @co = cc
        @res = e
        @st.call
      }
    }
    @res = nil
    @st.call
  end
  def next
    callcc { |cc|
      @st = cc
      @co ? @co.call : nextco
    }
    return @res
  end
end

a = [1,2,3]
b = [4,5,6,7,8]

while x = a.next and y = b.next
  puts x,y
end

Note there are three possible behaviours around the @res = nil
line. One is to return nil after all items have been used up, another
is to keep returning the last item by removing that line, and finally
a loop can be wrapped around the each, to begin returning from
the first item again.

Its probably possible to do it in one function, but it makes more
sense to me in two.

Yes I understand that I take a hit (wrt perf by choosing ruby), but
that's a BAD argument to favour wrapping state first by a iterators and
then once over by a continuation.

As long as its wrapped safely and nicely, what does it matter?
This way, you even get your choice of what to do when you
run out of items :wink:

···

On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:

--
spooq

4

Gah, always miss something when I post code...

Replace the first two lines of nextco with
  def nextco
    each_with_index { |e,i|

···

On Fri, 11 Mar 2005 23:41:46 +1000, Luke Graham <spoooq@gmail.com> wrote:

On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:

> I was trying to write code where I can have two or more computations
> which generate values that I need to compare - with the ruby syntax
> tying iterator usage to the code blocks by syntax, there is no way I can
> do this. This is especially applicable when the iterators potentially
> return an infinite stream of values. (Except for using contiuations...
> Which is not really so much of a solution, because I might as well not
> have used an iterator in a first place if I could create good
> coroutines)

Continuations are the right answer in this case.

module Enumerable
        attr_accessor :co, :st, :res

        def nextco
                each { |e|
                        callcc { |cc|
                                @co = cc
                                @res = e
                                @st.call
                        }
                }
                @res = nil
                @st.call
        end
        def next
                callcc { |cc|
                        @st = cc
                        @co ? @co.call : nextco
                }
                return @res
        end
end

a = [1,2,3]
b = [4,5,6,7,8]

while x = a.next and y = b.next
        puts x,y
end

Note there are three possible behaviours around the @res = nil
line. One is to return nil after all items have been used up, another
is to keep returning the last item by removing that line, and finally
a loop can be wrapped around the each, to begin returning from
the first item again.

Its probably possible to do it in one function, but it makes more
sense to me in two.

> Yes I understand that I take a hit (wrt perf by choosing ruby), but
> that's a BAD argument to favour wrapping state first by a iterators and
> then once over by a continuation.

As long as its wrapped safely and nicely, what does it matter?
This way, you even get your choice of what to do when you
run out of items :wink:

--
spooq

--
spooq

5

Just found this, it seems to fit the bill nicely and its in the std
lib of all places :stuck_out_tongue:

http://www.ruby-doc.org/stdlib/libdoc/generator/rdoc/classes/Generator.html

···

On Fri, 11 Mar 2005 23:44:39 +1000, Luke Graham <spoooq@gmail.com> wrote:

Gah, always miss something when I post code...

Replace the first two lines of nextco with
        def nextco
                each_with_index { |e,i|

On Fri, 11 Mar 2005 23:41:46 +1000, Luke Graham <spoooq@gmail.com> wrote:
> On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:
>
> > I was trying to write code where I can have two or more computations
> > which generate values that I need to compare - with the ruby syntax
> > tying iterator usage to the code blocks by syntax, there is no way I can
> > do this. This is especially applicable when the iterators potentially
> > return an infinite stream of values. (Except for using contiuations...
> > Which is not really so much of a solution, because I might as well not
> > have used an iterator in a first place if I could create good
> > coroutines)
>
> Continuations are the right answer in this case.
>
> module Enumerable
> attr_accessor :co, :st, :res
>
> def nextco
> each { |e|
> callcc { |cc|
> @co = cc
> @res = e
> @st.call
> }
> }
> @res = nil
> @st.call
> end
> def next
> callcc { |cc|
> @st = cc
> @co ? @co.call : nextco
> }
> return @res
> end
> end
>
> a = [1,2,3]
> b = [4,5,6,7,8]
>
> while x = a.next and y = b.next
> puts x,y
> end
>
> Note there are three possible behaviours around the @res = nil
> line. One is to return nil after all items have been used up, another
> is to keep returning the last item by removing that line, and finally
> a loop can be wrapped around the each, to begin returning from
> the first item again.
>
> Its probably possible to do it in one function, but it makes more
> sense to me in two.
>
> > Yes I understand that I take a hit (wrt perf by choosing ruby), but
> > that's a BAD argument to favour wrapping state first by a iterators and
> > then once over by a continuation.
>
> As long as its wrapped safely and nicely, what does it matter?
> This way, you even get your choice of what to do when you
> run out of items :wink:
>
> --
> spooq
>

--
spooq

--
spooq

6

More or less compliant with the generator.rb api, which I like.
Some proof that the thing works, too.

module Enumerable
  attr_accessor :co, :st, :res, :sto
  
  def nextco
    each_with_index { |e,i|
      callcc { |cc|
        @co = cc
        @res = e
        @st.call
      }
    }
    @res = nil
    @st.call
  end
  def next
    (t = @sto; @sto = nil; return t) if @sto
    callcc { |cc|
      @st = cc
      @co ? @co.call : nextco
    }
    return @res
  end
  def next?
    @sto = self.next if !@sto
    @sto
  end
end

a = [1,2,4]
b = [3,5,'lots']

puts a.next
b.next?
b.next?
b.next?

while a.next? and b.next?
  puts a.next,b.next
end

···

On Fri, 11 Mar 2005 23:58:30 +1000, Luke Graham <spoooq@gmail.com> wrote:

Just found this, it seems to fit the bill nicely and its in the std
lib of all places :stuck_out_tongue:

http://www.ruby-doc.org/stdlib/libdoc/generator/rdoc/classes/Generator.html

On Fri, 11 Mar 2005 23:44:39 +1000, Luke Graham <spoooq@gmail.com> wrote:
> Gah, always miss something when I post code...
>
> Replace the first two lines of nextco with
> def nextco
> each_with_index { |e,i|
>
>
> On Fri, 11 Mar 2005 23:41:46 +1000, Luke Graham <spoooq@gmail.com> wrote:
> > On Fri, 11 Mar 2005 15:57:08 +0900, Roshan James <roshanj@microsoft.com> wrote:
> >
> > > I was trying to write code where I can have two or more computations
> > > which generate values that I need to compare - with the ruby syntax
> > > tying iterator usage to the code blocks by syntax, there is no way I can
> > > do this. This is especially applicable when the iterators potentially
> > > return an infinite stream of values. (Except for using contiuations...
> > > Which is not really so much of a solution, because I might as well not
> > > have used an iterator in a first place if I could create good
> > > coroutines)
> >
> > Continuations are the right answer in this case.
> >
> > module Enumerable
> > attr_accessor :co, :st, :res
> >
> > def nextco
> > each { |e|
> > callcc { |cc|
> > @co = cc
> > @res = e
> > @st.call
> > }
> > }
> > @res = nil
> > @st.call
> > end
> > def next
> > callcc { |cc|
> > @st = cc
> > @co ? @co.call : nextco
> > }
> > return @res
> > end
> > end
> >
> > a = [1,2,3]
> > b = [4,5,6,7,8]
> >
> > while x = a.next and y = b.next
> > puts x,y
> > end
> >
> > Note there are three possible behaviours around the @res = nil
> > line. One is to return nil after all items have been used up, another
> > is to keep returning the last item by removing that line, and finally
> > a loop can be wrapped around the each, to begin returning from
> > the first item again.
> >
> > Its probably possible to do it in one function, but it makes more
> > sense to me in two.
> >
> > > Yes I understand that I take a hit (wrt perf by choosing ruby), but
> > > that's a BAD argument to favour wrapping state first by a iterators and
> > > then once over by a continuation.
> >
> > As long as its wrapped safely and nicely, what does it matter?
> > This way, you even get your choice of what to do when you
> > run out of items :wink:
> >
> > --
> > spooq
> >
>
> --
> spooq
>

--
spooq

--
spooq

7

More or less compliant with the generator.rb api, which I like.
Some proof that the thing works, too.

module Enumerable
  attr_accessor :co, :st, :res, :sto
  
(snip)

end

I also wrote some lightweight generator, before I knew that there is one
in the std lib.

Now I see that it's about the same speed as your Enumerable extension,
ie., they are cca. 1.5 times faster than the official one.

This is enough reason to show it... it converts an iterator method to a
generator. Eg.:

require 'iter2gener'

a = (0...3000).to_a
g = I2G.new a.method(:each)
while e=g.next; p e; end

You have some further options, eg, you can set what to do when iterator
is exhausted.

Csaba

iter2gener.rb (423 Bytes)

···

On Fri, Mar 11, 2005 at 11:13:54PM +0900, Luke Graham wrote: