Enumerators and generators

ruby-talk
1

Hi, all...

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

I've used generators, but enumerators are newer to me.

I've heard one person's take on this (thanks David!) but thought
I might ask more thoughts...

Thanks,
Hal Fulton

2

Hello Hal

Generators "externalize" iteration, whereas Enumerator "internalize"
it (but this docs may have already told you).

Generators can be used to create a "controlled stream" of objects,
that you can operate as you where using a remove control: go to the
next (Generator#next), give me the position (Generator#pos), return to
the beginning (Generator#rewind), ...

Enumerator offers methods to iterate through the entire stream, like
iterate using a windows of size X (Enumerator#each_cons), iterate
using slices from the stream (Enumerator#each_slide), ...

They are quite different in use, although both are usable to iterate over data.

···

On Wed, Jul 21, 2010 at 22:12, Hal Fulton <rubyhacker@gmail.com> wrote:

Hi, all...

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

I've used generators, but enumerators are newer to me.

I've heard one person's take on this (thanks David!) but thought
I might ask more thoughts...

Thanks,
Hal Fulton

3

It seems the alternative is Generator vs. Enumerable rather than
Generator vs. Enumerator. Enumerator is rather a special form of
Enumerable which allows to do some things more efficiently because it
delays the iteration. E.g.

# traditional one liner style: needs much memory:
matches = File.readlines("foo").select {|line| /keyword/ =~ line}

# Enumerator: only memory for the
matches = File.to_enum(:foreach, "foo").select {|line| /keyword/ =~ line}

# Same in 1.9 with implicit Enumerator creation
matches = File.foreach("foo").select {|line| /keyword/ =~ line}

It's a convenient replacement for the equally efficient (memory wise):

matches =

File.foreach "foo" do |line|
  matches << line if /keyword/ =~ line
end

Generator is a completely different iteration style as Ricardo explained.

Kind regards

robert

···

2010/7/22 Hal Fulton <rubyhacker@gmail.com>:

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

4

Hal Fulton wrote:

Hi, all...

I'm trying to grasp when/why one might use a generator instead of
an enumerator (or vice versa).

I've used generators, but enumerators are newer to me.

Basically what was a Generator in 1.8 is now an enumerator in 1.9, and
faster.

http://wiki.github.com/rdp/ruby_tutorials_core/enumerator

Cheers!
-r

···

--
Posted via http://www.ruby-forum.com/\.

5

Hi --

Hello Hal

Generators "externalize" iteration, whereas Enumerator "internalize"
it (but this docs may have already told you).

Generators can be used to create a "controlled stream" of objects,
that you can operate as you where using a remove control: go to the
next (Generator#next), give me the position (Generator#pos), return to
the beginning (Generator#rewind), ...

Enumerators have #next and #rewind too, though (though not #pos). In
1.9, as far as I can tell, the generator.rb library has been removed,
and Generator is now a class inside Enumerator. An Enumerator::Generator
is created automatically, for the use of the enumerator, if you create
an enumerator with a block. Also, these Generators don't have #next and
friends; that's available via the enumerator. (I'm not sure what
happened to #pos.)

As I understand it, the main thing about generators is that "controlled
stream" thing, where you can roll your own sense of iteration, rather
than just piggy-backing on what some enumerable object with its own
ideas about iteration thinks. Here's a (very contrived) 1.9 example:

[dblack@ruby-versions ~]$ cat e.rb message = nil

g = Enumerator::Generator.new do |yielder|
   yielder << "Hi."
   puts "I've been told to #{message}."
   case message
   when "leave"
     yielder << "Bye."
   when "stay"
     yielder << "I'm still here!"
   end
end

e = Enumerator.new(g)

puts e.next
message = "stay"
puts e.next

e.rewind

puts e.next
message = "leave"
puts e.next

[dblack@ruby-versions ~]$ ruby e.rb Hi.
I've been told to stay.
I'm still here!
Hi.
I've been told to leave.
Bye.

David

···

On Thu, 22 Jul 2010, Ricardo Panaggio wrote:

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

   The Ruby training with Black/Brown/McAnally
   Compleat Philadelphia, PA, October 1-2, 2010
   Rubyist http://www.compleatrubyist.com

6

Hi --

Hello Hal

Generators "externalize" iteration, whereas Enumerator "internalize"
it (but this docs may have already told you).

Generators can be used to create a "controlled stream" of objects,
that you can operate as you where using a remove control: go to the
next (Generator#next), give me the position (Generator#pos), return to
the beginning (Generator#rewind), ...

Enumerators have #next and #rewind too, though (though not #pos). In
1.9, as far as I can tell, the generator.rb library has been removed,

I wasn't aware of it in 1.9. Good to now :slight_smile:

···

On Thu, Jul 22, 2010 at 07:11, David A. Black <dblack@rubypal.com> wrote:

On Thu, 22 Jul 2010, Ricardo Panaggio wrote:

and Generator is now a class inside Enumerator. An Enumerator::Generator
is created automatically, for the use of the enumerator, if you create
an enumerator with a block. Also, these Generators don't have #next and
friends; that's available via the enumerator. (I'm not sure what
happened to #pos.)

As I understand it, the main thing about generators is that "controlled
stream" thing, where you can roll your own sense of iteration, rather
than just piggy-backing on what some enumerable object with its own
ideas about iteration thinks. Here's a (very contrived) 1.9 example:

[dblack@ruby-versions ~]$ cat e.rb message = nil

g = Enumerator::Generator.new do |yielder|
yielder << "Hi."
puts "I've been told to #{message}."
case message
when "leave"
yielder << "Bye."
when "stay"
yielder << "I'm still here!"
end
end

e = Enumerator.new(g)

puts e.next
message = "stay"
puts e.next

e.rewind

puts e.next
message = "leave"
puts e.next

[dblack@ruby-versions ~]$ ruby e.rb Hi.
I've been told to stay.
I'm still here!
Hi.
I've been told to leave.
Bye.

David

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

The Ruby training with Black/Brown/McAnally
Compleat Philadelphia, PA, October 1-2, 2010
Rubyist http://www.compleatrubyist.com

7

My next questions then are:

1. Does Enumerator::Generator have all the functionality of the
old generator.rb, just (it seems to me) a different usage?

2. In David's example, is "yielder" simply an array of objects? Do
we know or care? (If so, I'd be tempted to call it "list" because
"yielder" reminds of the keyword yield which I suppose is unrelated
here.)

Hal

···

On Thu, Jul 22, 2010 at 7:59 AM, Ricardo Panaggio < panaggio.ricardo@gmail.com> wrote:

On Thu, Jul 22, 2010 at 07:11, David A. Black <dblack@rubypal.com> wrote:
> Hi --
>
> On Thu, 22 Jul 2010, Ricardo Panaggio wrote:
>
>> Hello Hal
>>
>> Generators "externalize" iteration, whereas Enumerator "internalize"
>> it (but this docs may have already told you).
>>
>> Generators can be used to create a "controlled stream" of objects,
>> that you can operate as you where using a remove control: go to the
>> next (Generator#next), give me the position (Generator#pos), return to
>> the beginning (Generator#rewind), ...
>
> Enumerators have #next and #rewind too, though (though not #pos). In
> 1.9, as far as I can tell, the generator.rb library has been removed,

I wasn't aware of it in 1.9. Good to now :slight_smile:

> and Generator is now a class inside Enumerator. An Enumerator::Generator
> is created automatically, for the use of the enumerator, if you create
> an enumerator with a block. Also, these Generators don't have #next and
> friends; that's available via the enumerator. (I'm not sure what
> happened to #pos.)
>
> As I understand it, the main thing about generators is that "controlled
> stream" thing, where you can roll your own sense of iteration, rather
> than just piggy-backing on what some enumerable object with its own
> ideas about iteration thinks. Here's a (very contrived) 1.9 example:
>
> [dblack@ruby-versions ~]$ cat e.rb message = nil
>
> g = Enumerator::Generator.new do |yielder|
> yielder << "Hi."
> puts "I've been told to #{message}."
> case message
> when "leave"
> yielder << "Bye."
> when "stay"
> yielder << "I'm still here!"
> end
> end
>
> e = Enumerator.new(g)
>
> puts e.next
> message = "stay"
> puts e.next
>
> e.rewind
>
> puts e.next
> message = "leave"
> puts e.next
>
> [dblack@ruby-versions ~]$ ruby e.rb Hi.
> I've been told to stay.
> I'm still here!
> Hi.
> I've been told to leave.
> Bye.
>
>
> David
>
> --
> David A. Black, Senior Developer, Cyrus Innovation Inc.
>
> The Ruby training with Black/Brown/McAnally
> Compleat Philadelphia, PA, October 1-2, 2010
> Rubyist http://www.compleatrubyist.com
>
>

8

Hi --

My next questions then are:

1. Does Enumerator::Generator have all the functionality of the
old generator.rb, just (it seems to me) a different usage?

No; it doesn't have #next or #rewind, but enumerators
(which can easily/transparently wrap generators) do. Neither of them
appears to have #pos, unless it's available in some other way that I
haven't spotted. (I'm writing this in haste and without benefit of irb,
so I'm not digging further right now.)

2. In David's example, is "yielder" simply an array of objects? Do
we know or care? (If so, I'd be tempted to call it "list" because
"yielder" reminds of the keyword yield which I suppose is unrelated
here.)

It's actually an Enumerator::Yielder object, which I think only exists
for the purpose of being the receptacle for the objects to be yielded by
the generator.

David

···

On Fri, 23 Jul 2010, Hal Fulton wrote:

--
David A. Black, Senior Developer, Cyrus Innovation Inc.

   The Ruby training with Black/Brown/McAnally
   Compleat Philadelphia, PA, October 1-2, 2010
   Rubyist http://www.compleatrubyist.com