Ruby needs continuations

ruby-talk
1

Warning: I don't really know what I'm talking about so if I make any
mistakes in terminology, please try to correct me...

    I've been doing a lot of Python programming and I've discovered that
it's actually a very powerful language. The language, itself, lacks any
kind of elegance but it has all the power of Ruby and a little more
performance. It also has a richer set of libraries, although not in all
areas, surprisingly.
    One thing that Ruby should take from Python are continuations. Python
is moving away from list creation and version 3.0 functions will return
Python iterators, implemented with continuations, instead of actual lists,
since they are not used nearly as much as you might think.
    The reason why I say that Ruby needs continuations is because they are
more versatile than Ruby iterators. The reason why I say _that_ is because
you can make Ruby iterators with continuations but you can't make
continuations with Ruby iterators. This means that you can implement a
continuation iterator and Object class can automatically define a Ruby
iterator based on your continuation iterator. After all, Ruby iterators are
nice. I'm surprised by how annoyed I am that Python for loops don't return
a value...
    One weakness of Ruby iterators that continuations don't have is parallel
iteration. If you have two containers that represent different aspects of
the same things, it's difficult to iterate over both of them in Ruby. In
Python, you can do this:

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

# Okay, Python is more wordy than I'd like
# I could have just used zip() but you rarely need the list!
for num, letter in itertools.izip(list1, list2):
    # Do something with both numbers and letters
    print num, letter
    print

    How would one do this in Ruby? You can use .each_with_index and index
the other list but that assumes that the other list is indexable, which only
happens to be true of arrays but is not true in general. I've thought about
implementing some Ruby equivalent of zip() (or preferably izip()) and
discovered that I can't do so without continuations.
    ...Hence, my post. I'm actually at odds with Pythonic philosophy. The
idea that there should only be one way to do things is ludicrous and a
constant up hill battle. One thing that Python does right is that it's not
afraid to "steal" from other languages and that's the right attitude to
have. Adopt whatever is useful!
    Python 3.0 looks like great language. I'm hoping that Ruby 2.0 will be
even better 'cause, frankly, Ruby is more fun to program in...
    Thank you...

2

I can't comment much on the rest of your post, considering I lack knowledge
both on the Python camp and the Ruby camp to make any meaningful
consideration.

Ruby does, however, have an implementation of zip in Enumerable. You can do

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

list1.zip(list2).each {|num, letter| puts "#{num} #{letter}" }

just fine. You'll actually get the same result you would with your Python
example.

Best regards,

Vitor

···

On Nov 28, 2007 10:30 AM, Just Another Victim of the Ambient Morality < ihatespam@hotmail.com> wrote:

   How would one do this in Ruby? You can use .each_with_index and index
the other list but that assumes that the other list is indexable, which
only
happens to be true of arrays but is not true in general. I've thought
about
implementing some Ruby equivalent of zip() (or preferably izip()) and
discovered that I can't do so without continuations.

3

[1,2,3].zip([4,5,6]).each{|a,b| p a => b}

also, ruby does have contiuations, check your facts (ri callcc) before
trying to complain, otherwise you will just appear as yet another
troll.

^ manveru

···

On 11/28/07, Just Another Victim of the Ambient Morality <ihatespam@hotmail.com> wrote:

Warning: I don't really know what I'm talking about so if I make any
mistakes in terminology, please try to correct me...

    I've been doing a lot of Python programming and I've discovered that
it's actually a very powerful language. The language, itself, lacks any
kind of elegance but it has all the power of Ruby and a little more
performance. It also has a richer set of libraries, although not in all
areas, surprisingly.
    One thing that Ruby should take from Python are continuations. Python
is moving away from list creation and version 3.0 functions will return
Python iterators, implemented with continuations, instead of actual lists,
since they are not used nearly as much as you might think.
    The reason why I say that Ruby needs continuations is because they are
more versatile than Ruby iterators. The reason why I say _that_ is because
you can make Ruby iterators with continuations but you can't make
continuations with Ruby iterators. This means that you can implement a
continuation iterator and Object class can automatically define a Ruby
iterator based on your continuation iterator. After all, Ruby iterators are
nice. I'm surprised by how annoyed I am that Python for loops don't return
a value...
    One weakness of Ruby iterators that continuations don't have is parallel
iteration. If you have two containers that represent different aspects of
the same things, it's difficult to iterate over both of them in Ruby. In
Python, you can do this:

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

# Okay, Python is more wordy than I'd like
# I could have just used zip() but you rarely need the list!
for num, letter in itertools.izip(list1, list2):
    # Do something with both numbers and letters
    print num, letter
    print

    How would one do this in Ruby? You can use .each_with_index and index
the other list but that assumes that the other list is indexable, which only
happens to be true of arrays but is not true in general. I've thought about
implementing some Ruby equivalent of zip() (or preferably izip()) and
discovered that I can't do so without continuations.
    ...Hence, my post. I'm actually at odds with Pythonic philosophy. The
idea that there should only be one way to do things is ludicrous and a
constant up hill battle. One thing that Python does right is that it's not
afraid to "steal" from other languages and that's the right attitude to
have. Adopt whatever is useful!
    Python 3.0 looks like great language. I'm hoping that Ruby 2.0 will be
even better 'cause, frankly, Ruby is more fun to program in...
    Thank you...

4

Ruby has continuations. Though in practice they are not used much.
They have caused serious pain in implementing JRuby for instance,
to the point that IIRC JRuby does not support them out of the box (and
this doesn't diminish JRuby's value really).

But there are a lot of really smart people looking into Ruby right now.
Don't expect the dormant Lisp-like features to stay unused.

Python 'lacks any kind of elegance'? Thats unfair. It is more correct to
say, that Python has a different value system for code expressiveness.

Python and Ruby are more or less peers on the Language Power Continuum.
Choice between either will depend more on specific task details, specific
library availability or personal preference.

···

On Nov 28, 2007 12:30 PM, Just Another Victim of the Ambient Morality <ihatespam@hotmail.com> wrote:

Warning: I don't really know what I'm talking about so if I make any
mistakes in terminology, please try to correct me...

    I've been doing a lot of Python programming and I've discovered that
it's actually a very powerful language. The language, itself, lacks any
kind of elegance but it has all the power of Ruby and a little more
performance. It also has a richer set of libraries, although not in all
areas, surprisingly.
    One thing that Ruby should take from Python are continuations.

5

You actually need a generator, which can be implemented with
continuations, but doesn't have to be (in 1.9 the generators are
implemented with threads -- have a look at generator.rb in 1.8 and 1.9
to see the differences). Using generators, you can implement izip (and
all of itertools if you'd like, though most of it already has
equivalent ruby solutions). Here is a pretty close translation of the
python given in the itertools docs:

···

On Nov 28, 6:29 am, "Just Another Victim of the Ambient Morality" <ihates...@hotmail.com> wrote:

Warning: I don't really know what I'm talking about so if I make any
mistakes in terminology, please try to correct me...

    I've been doing a lot of Python programming and I've discovered that
it's actually a very powerful language. The language, itself, lacks any
kind of elegance but it has all the power of Ruby and a little more
performance. It also has a richer set of libraries, although not in all
areas, surprisingly.
    One thing that Ruby should take from Python are continuations. Python
is moving away from list creation and version 3.0 functions will return
Python iterators, implemented with continuations, instead of actual lists,
since they are not used nearly as much as you might think.
    The reason why I say that Ruby needs continuations is because they are
more versatile than Ruby iterators. The reason why I say _that_ is because
you can make Ruby iterators with continuations but you can't make
continuations with Ruby iterators. This means that you can implement a
continuation iterator and Object class can automatically define a Ruby
iterator based on your continuation iterator. After all, Ruby iterators are
nice. I'm surprised by how annoyed I am that Python for loops don't return
a value...
    One weakness of Ruby iterators that continuations don't have is parallel
iteration. If you have two containers that represent different aspects of
the same things, it's difficult to iterate over both of them in Ruby. In
Python, you can do this:

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

# Okay, Python is more wordy than I'd like
# I could have just used zip() but you rarely need the list!
for num, letter in itertools.izip(list1, list2):
    # Do something with both numbers and letters
    print num, letter
    print

    How would one do this in Ruby? You can use .each_with_index and index
the other list but that assumes that the other list is indexable, which only
happens to be true of arrays but is not true in general. I've thought about
implementing some Ruby equivalent of zip() (or preferably izip()) and
discovered that I can't do so without continuations.
    ...Hence, my post. I'm actually at odds with Pythonic philosophy. The
idea that there should only be one way to do things is ludicrous and a
constant up hill battle. One thing that Python does right is that it's not
afraid to "steal" from other languages and that's the right attitude to
have. Adopt whatever is useful!
    Python 3.0 looks like great language. I'm hoping that Ruby 2.0 will be
even better 'cause, frankly, Ruby is more fun to program in...
    Thank you...

========

require 'generator'

module Enumerable
  def izip(*enumerables)
    enumerables = [self] + enumerables
    generators = enumerables.map { | enum |
      Generator.new(enum)
    }
    while generators[0].next?
      result = generators.map { | gen |
        gen.next
      }
      yield result
    end
  end
end

[1,2,3].izip([4,5,6]) { | x, y |
  puts x, y
}

========

NB. generator.rb says that generators are slow in 1.8.

Regards,
Jordan

6

At RubyConf 2007 last month, Matz announced that 1.9 would have
external iterators. That would allow you to do this:

num_iter = list1.each
let_iter = list2.each
loop {
  puts num_iter.next, let_iter.next
}

When you reach the end of either list (or any list, if we scale beyond
just two), the call to next would raise an exception, automatically
caught by the loop construct, causing the loop to simply exit. So
this construct works gracefully even if one of the lists is shorter
than the other(s).

Eric

P.S. I put together a very short presentation for the Southeast
Michigan Ruby Users' Group summarizing what Matz said about 1.9. You
can see the slides at:

    http://learnruby.com/ruby-1.9.html

···

On Nov 28, 7:29 am, "Just Another Victim of the Ambient Morality" <ihates...@hotmail.com> wrote:

    One weakness of Ruby iterators that continuations don't have is parallel
iteration. If you have two containers that represent different aspects of
the same things, it's difficult to iterate over both of them in Ruby. In
Python, you can do this:

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

# Okay, Python is more wordy than I'd like
# I could have just used zip() but you rarely need the list!
for num, letter in itertools.izip(list1, list2):
    # Do something with both numbers and letters
    print num, letter
    print

    How would one do this in Ruby?

====

Are you interested in on-site Ruby training that uses well-designed,
real-world, hands-on exercises? http://LearnRuby.com

7

> How would one do this in Ruby? You can use .each_with_index and index
> the other list but that assumes that the other list is indexable, which
> only happens to be true of arrays but is not true in general. I've thought
> about implementing some Ruby equivalent of zip() (or preferably izip()) and
> discovered that I can't do so without continuations.

Actually, Ruby 1.9 is losing continuations (hard to do with native
threads, expensive for stack frame manipulation in JRuby). However,
you don't need continuations to do parallel iteration, as Vitor
suggests:

Ruby does, however, have an implementation of zip in Enumerable. You can do

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

list1.zip(list2).each {|num, letter| puts "#{num} #{letter}" }

just fine. You'll actually get the same result you would with your Python
example.

You're also able to do:

  a.zip(b, c, d, ...)

to iterate multiple lists in parallel.

-austin

···

On 11/28/07, Vitor Peres <dodecaphonic@gmail.com> wrote:

On Nov 28, 2007 10:30 AM, Just Another Victim of the Ambient Morality < > ihatespam@hotmail.com> wrote:

--
Austin Ziegler * halostatue@gmail.com * http://www.halostatue.ca/
               * austin@halostatue.ca * You are in a maze of twisty little passages, all alike. // halo • statue
               * austin@zieglers.ca

8

"Michael Fellinger" <m.fellinger@gmail.com> wrote in message
news:9c00d3e00711280453l26abd03h9f5b8791ae699ef1@mail.gmail.com...

[1,2,3].zip([4,5,6]).each{|a,b| p a => b}

also, ruby does have contiuations, check your facts (ri callcc) before
trying to complain, otherwise you will just appear as yet another
troll.

    Correct me if I'm wrong but aren't continuations being dropped for Ruby
2.0?

9

"Richard Conroy" <richard.conroy@gmail.com> wrote in message
news:511fa3a20711280513m702a8b1ft2f0ee7f1130ee1be@mail.gmail.com...

Warning: I don't really know what I'm talking about so if I make any
mistakes in terminology, please try to correct me...

    I've been doing a lot of Python programming and I've discovered that
it's actually a very powerful language. The language, itself, lacks any
kind of elegance but it has all the power of Ruby and a little more
performance. It also has a richer set of libraries, although not in all
areas, surprisingly.
    One thing that Ruby should take from Python are continuations.

Ruby has continuations. Though in practice they are not used much.
They have caused serious pain in implementing JRuby for instance,
to the point that IIRC JRuby does not support them out of the box (and
this doesn't diminish JRuby's value really).

But there are a lot of really smart people looking into Ruby right now.
Don't expect the dormant Lisp-like features to stay unused.

Python 'lacks any kind of elegance'? Thats unfair. It is more correct to
say, that Python has a different value system for code expressiveness.

    Indeed, I wasn't trying to be fair, I was just expressing a personal
preference. While Python is clearly a powerful and useful language (I'm
programming in it, aren't I?), its syntax choices irk me to no end. len()
is a function that can take a list but .append() is a method of list's?
Arg! Zero and empty strings evaluate to false? Arg! Both zip() and
dict.items() are built-in but izip() hides in the itertools module while
dict.itertimes() gets to be built-in? Arg! Considering how iterators are
more useful than actual lists, double arg!
    As it so happened, I did expand on some of the differences between
Python's code values and mine, later on in my post...

Python and Ruby are more or less peers on the Language Power Continuum.
Choice between either will depend more on specific task details, specific
library availability or personal preference.

    Of course. I'm programming in Python, aren't I?

···

On Nov 28, 2007 12:30 PM, Just Another Victim of the Ambient Morality > <ihatespam@hotmail.com> wrote:

10

Ps. The SyncEnumerator class from generator does the same thing as
izip:

SyncEnumerator.new([1,2,3], [4,5,6]).each { | x, y |
  puts x, y
}

···

On Nov 28, 9:19 am, MonkeeSage <MonkeeS...@gmail.com> wrote:

On Nov 28, 6:29 am, "Just Another Victim of the Ambient Morality" > > > > <ihates...@hotmail.com> wrote:
> Warning: I don't really know what I'm talking about so if I make any
> mistakes in terminology, please try to correct me...

> I've been doing a lot of Python programming and I've discovered that
> it's actually a very powerful language. The language, itself, lacks any
> kind of elegance but it has all the power of Ruby and a little more
> performance. It also has a richer set of libraries, although not in all
> areas, surprisingly.
> One thing that Ruby should take from Python are continuations. Python
> is moving away from list creation and version 3.0 functions will return
> Python iterators, implemented with continuations, instead of actual lists,
> since they are not used nearly as much as you might think.
> The reason why I say that Ruby needs continuations is because they are
> more versatile than Ruby iterators. The reason why I say _that_ is because
> you can make Ruby iterators with continuations but you can't make
> continuations with Ruby iterators. This means that you can implement a
> continuation iterator and Object class can automatically define a Ruby
> iterator based on your continuation iterator. After all, Ruby iterators are
> nice. I'm surprised by how annoyed I am that Python for loops don't return
> a value...
> One weakness of Ruby iterators that continuations don't have is parallel
> iteration. If you have two containers that represent different aspects of
> the same things, it's difficult to iterate over both of them in Ruby. In
> Python, you can do this:

> list1 = [1, 2, 3, 4]
> list2 = ['a', 'b', 'c', 'd']

> # Okay, Python is more wordy than I'd like
> # I could have just used zip() but you rarely need the list!
> for num, letter in itertools.izip(list1, list2):
> # Do something with both numbers and letters
> print num, letter
> print

> How would one do this in Ruby? You can use .each_with_index and index
> the other list but that assumes that the other list is indexable, which only
> happens to be true of arrays but is not true in general. I've thought about
> implementing some Ruby equivalent of zip() (or preferably izip()) and
> discovered that I can't do so without continuations.
> ...Hence, my post. I'm actually at odds with Pythonic philosophy. The
> idea that there should only be one way to do things is ludicrous and a
> constant up hill battle. One thing that Python does right is that it's not
> afraid to "steal" from other languages and that's the right attitude to
> have. Adopt whatever is useful!
> Python 3.0 looks like great language. I'm hoping that Ruby 2.0 will be
> even better 'cause, frankly, Ruby is more fun to program in...
> Thank you...

You actually need a generator, which can be implemented with
continuations, but doesn't have to be (in 1.9 the generators are
implemented with threads -- have a look at generator.rb in 1.8 and 1.9
to see the differences). Using generators, you can implement izip (and
all of itertools if you'd like, though most of it already has
equivalent ruby solutions). Here is a pretty close translation of the
python given in the itertools docs:

========

require 'generator'

module Enumerable
  def izip(*enumerables)
    enumerables = [self] + enumerables
    generators = enumerables.map { | enum |
      Generator.new(enum)
    }
    while generators[0].next?
      result = generators.map { | gen |
        gen.next
      }
      yield result
    end
  end
end

[1,2,3].izip([4,5,6]) { | x, y |
  puts x, y

}

========

NB. generator.rb says that generators are slow in 1.8.

Regards,
Jordan

11

MonkeeSage wrote:

You actually need a generator, which can be implemented with
continuations, but doesn't have to be (in 1.9 the generators are
implemented with threads -- have a look at generator.rb in 1.8 and 1.9
to see the differences). Using generators, you can implement izip (and
all of itertools if you'd like, though most of it already has
equivalent ruby solutions). Here is a pretty close translation of the
python given in the itertools docs:

Ruby 1.9 also moves continuations out of core, but provides a form of bounded continuations (coroutines) that's possible to implement in JRuby:

require 'fiber'
f = Fiber.new { a = 1; while true; Fiber.yield a; a += 1; end }
5.times { puts f.resume }

$ jruby -J-Djruby.compat.version=ruby1_9 fiber_example.rb
1
2
3
4
5

Under JRuby, it's using a native thread per live Fiber, so it's heavier than in Ruby 1.9 which uses green threads. However they'll actually run in parallel on JRuby, so that's a benefit. JRuby also supports a thread pool in 1.1 that helps blunt the cost of spinning up native threads.

- Charlie

12

Cool! I just tried that, and they're already in there (1.9.0
patchlevel 0). :slight_smile:

Regards,
Jordan

···

On Nov 28, 12:08 pm, "Eric I." <rubytrain...@gmail.com> wrote:

On Nov 28, 7:29 am, "Just Another Victim of the Ambient Morality" > > > > <ihates...@hotmail.com> wrote:
> One weakness of Ruby iterators that continuations don't have is parallel
> iteration. If you have two containers that represent different aspects of
> the same things, it's difficult to iterate over both of them in Ruby. In
> Python, you can do this:

> list1 = [1, 2, 3, 4]
> list2 = ['a', 'b', 'c', 'd']

> # Okay, Python is more wordy than I'd like
> # I could have just used zip() but you rarely need the list!
> for num, letter in itertools.izip(list1, list2):
> # Do something with both numbers and letters
> print num, letter
> print

> How would one do this in Ruby?

At RubyConf 2007 last month, Matz announced that 1.9 would have
external iterators. That would allow you to do this:

num_iter = list1.each
let_iter = list2.each
loop {
  puts num_iter.next, let_iter.next

}

When you reach the end of either list (or any list, if we scale beyond
just two), the call to next would raise an exception, automatically
caught by the loop construct, causing the loop to simply exit. So
this construct works gracefully even if one of the lists is shorter
than the other(s).

Eric

P.S. I put together a very short presentation for the Southeast
Michigan Ruby Users' Group summarizing what Matz said about 1.9. You
can see the slides at:

   http://learnruby.com/ruby-1.9.html

====

Are you interested in on-site Ruby training that uses well-designed,
real-world, hands-on exercises?http://LearnRuby.com

13

"Austin Ziegler" <halostatue@gmail.com> wrote in message
news:9e7db9110711280500j3d7b1c6byf36c6d769c05bcbd@mail.gmail.com...

> How would one do this in Ruby? You can use .each_with_index and
> index
> the other list but that assumes that the other list is indexable, which
> only happens to be true of arrays but is not true in general. I've
> thought
> about implementing some Ruby equivalent of zip() (or preferably izip())
> and
> discovered that I can't do so without continuations.

Actually, Ruby 1.9 is losing continuations (hard to do with native
threads, expensive for stack frame manipulation in JRuby). However,
you don't need continuations to do parallel iteration, as Vitor
suggests:

Ruby does, however, have an implementation of zip in Enumerable. You can
do

list1 = [1, 2, 3, 4]
list2 = ['a', 'b', 'c', 'd']

list1.zip(list2).each {|num, letter| puts "#{num} #{letter}" }

just fine. You'll actually get the same result you would with your Python
example.

You're also able to do:

a.zip(b, c, d, ...)

to iterate multiple lists in parallel.

    Thank you, both, for pointing this out!
    Considering my failure to create such a method, I was about to ask how
such a thing was implemented but then I just tried it out and I've
discovered how: arrays. I've already mentioned that not all containers are
indexable: linked lists are not and trees are not efficiently.
Continuations allow even non-indexable containers to be iterated in
parallel...

···

On 11/28/07, Vitor Peres <dodecaphonic@gmail.com> wrote:

On Nov 28, 2007 10:30 AM, Just Another Victim of the Ambient Morality < >> ihatespam@hotmail.com> wrote:

14

That's not entirely true. In Ruby 1.9, continuations have been moved out of the core but are still available from a standard library.

James Edward Gray II

···

On Nov 28, 2007, at 7:00 AM, Austin Ziegler wrote:

On 11/28/07, Vitor Peres <dodecaphonic@gmail.com> wrote:

On Nov 28, 2007 10:30 AM, Just Another Victim of the Ambient >> Morality < >> ihatespam@hotmail.com> wrote:

   How would one do this in Ruby? You can use .each_with_index and index
the other list but that assumes that the other list is indexable, which
only happens to be true of arrays but is not true in general. I've thought
about implementing some Ruby equivalent of zip() (or preferably izip()) and
discovered that I can't do so without continuations.

Actually, Ruby 1.9 is losing continuations (hard to do with native
threads, expensive for stack frame manipulation in JRuby).

15

this iterates a *single* freshly allocated array:

static VALUE
rb_ary_zip(argc, argv, ary)
     int argc;
     VALUE *argv;
     VALUE ary;
{
     int i, j;
     long len;
     VALUE result;

     for (i=0; i<argc; i++) {
  argv[i] = to_ary(argv[i]);
     }
     if (rb_block_given_p()) {
  for (i=0; i<RARRAY(ary)->len; i++) {
      VALUE tmp = rb_ary_new2(argc+1);

      rb_ary_push(tmp, rb_ary_elt(ary, i));
      for (j=0; j<argc; j++) {
    rb_ary_push(tmp, rb_ary_elt(argv[j], i));
      }
      rb_yield(tmp);
  }
  return Qnil;
     }
     len = RARRAY(ary)->len;
     result = rb_ary_new2(len);
     for (i=0; i<len; i++) {
  VALUE tmp = rb_ary_new2(argc+1);

  rb_ary_push(tmp, rb_ary_elt(ary, i));
  for (j=0; j<argc; j++) {
      rb_ary_push(tmp, rb_ary_elt(argv[j], i));
  }
  rb_ary_push(result, tmp);
     }
     return result;
}

when a, b, c, and d are large it's most undesirable to use zip, forcing one to use

lists = a,b,c,d
n = lists.map{|list| list.size}.max

(0 ... n).each do |i|
   ai, bi, ci, di = lists.map{|list| list[i]}
   # ...
end

and, when b, c, and d are heterogeneous containers the above code is most often impossible - you will first need to convert all the containers to lists and then iterate. for me this is a major gripe about ruby that i work around nearly every day as it turns out many algorithms, such as those in image processing, revolve around iterating multiple containers at once: often in non-linear ways. one way around, without callcc, is just to use threads:

cfp:~ > cat a.rb
require 'thread'
module Enumerable
   def self.each *enumerables, &block
     n = enumerables.size
     done = Object.new
     qs = Array.new(n){ SizedQueue.new 1 }
     ts = Array.new(n){ |i|
       Thread.new{
         Thread.current.abort_on_exception = true
         e, q = enumerables[i], qs[i]
         e.each{|x| q.push x}
         q.push done
       }
     }
     dead = Array.new(n){ false }
     loop do
       values =
       finished = true
       qs.each_with_index do |q,i|
         if dead[i]
           values << nil
           next
         end
         value = q.pop
         if value == done
           dead[i] = true
           values << nil
         else
           values << value
         end
       end
       return enumerables if dead.all?
       block.call *values
     end
   end
end

a = 0,1,2
b = 2,3,4
c = 5,6,7

lists = a,b,c

Enumerable.each(a,b,c){|x,y,z| p [x,y,z]}

cfp:~ > ruby a.rb
[0, 2, 5]
[1, 3, 6]
[2, 4, 7]

but this, i think we can all agree, is slow and fugly. esp compared to something like this

   http://kogs-www.informatik.uni-hamburg.de/~koethe/vigra/doc/vigra/group__ImageIterators.html

external iterators are quite useful imho.

kind regards.

a @ http://codeforpeople.com/

···

On Nov 28, 2007, at 6:00 AM, Austin Ziegler wrote:

  a.zip(b, c, d, ...)

--
it is not enough to be compassionate. you must act.
h.h. the 14th dalai lama

16

"MonkeeSage" <MonkeeSage@gmail.com> wrote in message
news:4f89dee4-b922-49ef-889f-e281e7ad0e8c@j44g2000hsj.googlegroups.com...

···

On Nov 28, 9:19 am, MonkeeSage <MonkeeS...@gmail.com> wrote:

========

require 'generator'

module Enumerable
  def izip(*enumerables)
    enumerables = [self] + enumerables
    generators = enumerables.map { | enum |
      Generator.new(enum)
    }
    while generators[0].next?
      result = generators.map { | gen |
        gen.next
      }
      yield result
    end
  end
end

[1,2,3].izip([4,5,6]) { | x, y |
  puts x, y

}

========

NB. generator.rb says that generators are slow in 1.8.

Regards,
Jordan

Ps. The SyncEnumerator class from generator does the same thing as
izip:

SyncEnumerator.new([1,2,3], [4,5,6]).each { | x, y |
puts x, y
}

    Thank you, MonkeeSage, this is exactly what I'm looking for!
    It's interesting that generators are slow enough to warn users about its
lack of speed! Do you know if it would be any faster implemented with
continuations? I'm surprised it's so slow considering Ruby employs green
threads...

17

I just tried the above izip function in jruby, but it only gave me two
iterations...

$ jruby test.rb
1
4
2
5

...hmm?

$ jruby -v
ruby 1.8.5 (2007-11-28 rev 3841) [i386-jruby1.0]

Regards,
Jordan

···

On Nov 28, 9:41 am, Charles Oliver Nutter <charles.nut...@sun.com> wrote:

MonkeeSage wrote:
> You actually need a generator, which can be implemented with
> continuations, but doesn't have to be (in 1.9 the generators are
> implemented with threads -- have a look at generator.rb in 1.8 and 1.9
> to see the differences). Using generators, you can implement izip (and
> all of itertools if you'd like, though most of it already has
> equivalent ruby solutions). Here is a pretty close translation of the
> python given in the itertools docs:

Ruby 1.9 also moves continuations out of core, but provides a form of
bounded continuations (coroutines) that's possible to implement in JRuby:

require 'fiber'
f = Fiber.new { a = 1; while true; Fiber.yield a; a += 1; end }
5.times { puts f.resume }

$ jruby -J-Djruby.compat.version=ruby1_9 fiber_example.rb
1
2
3
4
5

Under JRuby, it's using a native thread per live Fiber, so it's heavier
than in Ruby 1.9 which uses green threads. However they'll actually run
in parallel on JRuby, so that's a benefit. JRuby also supports a thread
pool in 1.1 that helps blunt the cost of spinning up native threads.

- Charlie

18

Other way around. :wink: The continuation version is slow. The threaded
version is fast.

···

On Nov 28, 10:53 am, "Just Another Victim of the Ambient Morality" <ihates...@hotmail.com> wrote:

"MonkeeSage" <MonkeeS...@gmail.com> wrote in message

news:4f89dee4-b922-49ef-889f-e281e7ad0e8c@j44g2000hsj.googlegroups.com...

> On Nov 28, 9:19 am, MonkeeSage <MonkeeS...@gmail.com> wrote:
>> ========

>> require 'generator'

>> module Enumerable
>> def izip(*enumerables)
>> enumerables = [self] + enumerables
>> generators = enumerables.map { | enum |
>> Generator.new(enum)
>> }
>> while generators[0].next?
>> result = generators.map { | gen |
>> gen.next
>> }
>> yield result
>> end
>> end
>> end

>> [1,2,3].izip([4,5,6]) { | x, y |
>> puts x, y

>> }

>> ========

>> NB. generator.rb says that generators are slow in 1.8.

>> Regards,
>> Jordan

> Ps. The SyncEnumerator class from generator does the same thing as
> izip:

> SyncEnumerator.new([1,2,3], [4,5,6]).each { | x, y |
> puts x, y
> }

    Thank you, MonkeeSage, this is exactly what I'm looking for!
    It's interesting that generators are slow enough to warn users about its
lack of speed! Do you know if it would be any faster implemented with
continuations? I'm surprised it's so slow considering Ruby employs green
threads...

19

ara.t.howard wrote:

  a.zip(b, c, d, ...)

when a, b, c, and d are large it's most undesirable to use zip, forcing one to use

lists = a,b,c,d
n = lists.map{|list| list.size}.max

(0 ... n).each do |i|
  ai, bi, ci, di = lists.map{|list| list[i]}
  # ...
end

zip doesn't force you to do this, you can also pass a block to zip. In Ruby 1.9. zip without a block won't create and return an array anymore, but it will return an Enumerator object as a promise instead.

and, when b, c, and d are heterogeneous containers the above code is most often impossible - you will first need to convert all the containers to lists and then iterate.

If you want zip to act smarter on it's arguments you will have to use Ruby 1.9.

···

--
Florian Frank

20

MonkeeSage wrote:

···

On Nov 28, 9:41 am, Charles Oliver Nutter <charles.nut...@sun.com> > wrote:

MonkeeSage wrote:

You actually need a generator, which can be implemented with
continuations, but doesn't have to be (in 1.9 the generators are
implemented with threads -- have a look at generator.rb in 1.8 and 1.9
to see the differences). Using generators, you can implement izip (and
all of itertools if you'd like, though most of it already has
equivalent ruby solutions). Here is a pretty close translation of the
python given in the itertools docs:

Ruby 1.9 also moves continuations out of core, but provides a form of
bounded continuations (coroutines) that's possible to implement in JRuby:

require 'fiber'
f = Fiber.new { a = 1; while true; Fiber.yield a; a += 1; end }
5.times { puts f.resume }

$ jruby -J-Djruby.compat.version=ruby1_9 fiber_example.rb
1
2
3
4
5

Under JRuby, it's using a native thread per live Fiber, so it's heavier
than in Ruby 1.9 which uses green threads. However they'll actually run
in parallel on JRuby, so that's a benefit. JRuby also supports a thread
pool in 1.1 that helps blunt the cost of spinning up native threads.

- Charlie

I just tried the above izip function in jruby, but it only gave me two
iterations...

Could be a bug in generator; could you try it on a more recent release (like 1.0.2 or 1.1b1) and report it if it's actually a problem?

http://jira.codehaus.org/browse/JRUBY

- Charlie