Avoiding capturing local variables in closures

ruby-talk
1

I do a lot of event-driven programming (Eventmachine, ZMQMachine). You would think that by now I would know how to answer this question but I'm a bit stumped.

I have a memory leak in a program. I am 80% sure that the leak is due to a long-lived block having captured a local variable (containing a *lot* of data). I would like to prevent this from happening, but I'm unsure of the best way to accomplish it.

I've read Ola Bini's blog article on this topic [1]. However, I'm still a bit unclear on the concept.

An example...

def some_callback(bigarg1, bigarg2)
  foo = transform(bigarg1)
  bar = transformagain(bigarg2)

  schedule_for_execution { baz(foo, bar) }
end

If I am understanding things correctly, the block passed to #schedule_for_execution is capturing the variables foo, bar, bigarg1 and bigarg2. I could avoid capturing bigarg1 and bigarg2 with a small refactoring.

def some_callback(bigarg1, bigarg2)
  foo = transform(bigarg1)
  bar = transformagain(bigarg2)

  avoid_capture(foo, bar)
end

def avoid_capture(foo, bar)
  schedule_for_execution { baz(foo, bar) }
end

Is this sufficient to avoid capturing those args in the closure? If not, what do I need to do?

Further, what if I have an instance var that is referencing some large data? Do I need to worry about that being captured by the block or does it not matter because the block has already captured self?

e.g.

def some_callback(bigarg1, bigarg2)
  foo = transform(bigarg1)
  bar = transformagain(bigarg2)
  @quxxo = bigarg1 + bigarg2

  schedule_for_execution { baz(foo, bar) }
end

cr

[1] http://ola-bini.blogspot.com/2007/12/ruby-closures-and-memory-usage.html

2

That should be sufficient - there's no more reference to some_callback's
environment where bigarg1 and bigarg2 are referenced. Another
approach might be, on 1.9:

def some_callback(bigarg1)
  foo = transform(bigarg1)
  schedule_for_execution { |;bigarg1| bad(foo) }
end

This is the only real reason I know of for using the Ruby 1.9 block-local syntax. As
long as the implementation is smart enough not to foolishly store bigarg1.

Michael Edgar
adgar@carboni.ca
http://carboni.ca/

···

On Jul 29, 2011, at 5:48 PM, Chuck Remes wrote:

def some_callback(bigarg1, bigarg2)
foo = transform(bigarg1)
bar = transformagain(bigarg2)

schedule_for_execution { baz(foo, bar) }
end

If I am understanding things correctly, the block passed to #schedule_for_execution is capturing the variables foo, bar, bigarg1 and bigarg2. I could avoid capturing bigarg1 and bigarg2 with a small refactoring.

def some_callback(bigarg1, bigarg2)
foo = transform(bigarg1)
bar = transformagain(bigarg2)

avoid_capture(foo, bar)
end

def avoid_capture(foo, bar)
schedule_for_execution { baz(foo, bar) }
end

Is this sufficient to avoid capturing those args in the closure? If not, what do I need to do?

3

<snip>

Answered by Michael

Further, what if I have an instance var that is referencing some large data? Do I need to worry >about that being captured by the block or does it not matter because the block has already >captured self?

At first sight you need to worry if you pass the block to a method
outside of the receiver because that will create a new reference to
"self".
As long as you pass the block around in the same object I do not see an issue.

HTH
Robert

···

[1] Ola Bini: Programming Language Synchronicity: Ruby closures and memory usage

--
I'm not against types, but I don't know of any type systems that
aren't a complete pain, so I still like dynamic typing.
--
Alain Kay

4

Generally care should be taken about the value of self when a long
lived block is created. Typically classes are better candidates than
instances.

Kind regards

robert

···

On Sat, Jul 30, 2011 at 10:14 AM, Robert Dober <robert.dober@gmail.com> wrote:

<snip>

Answered by Michael

Further, what if I have an instance var that is referencing some large data? Do I need to worry >about that being captured by the block or does it not matter because the block has already >captured self?

At first sight you need to worry if you pass the block to a method
outside of the receiver because that will create a new reference to
"self".
As long as you pass the block around in the same object I do not see an issue.

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

5

I want to be clear that I understand you. Here's some code that I think illustrates what you mean.

class A
  def some_callback(bigarg1, bigarg2)
   foo = transform(bigarg1)
   bar = transformagain(bigarg2)
    @another_instance = B.new

   avoid_capture(foo, bar)
  end

  def avoid_capture(foo, bar)
    # self (from an instance of A) is captured in the block that
    # is now held by @another_instance
   @another_instance.schedule_for_execution { baz(foo, bar) }
  end
end

Right?

cr

···

On Jul 30, 2011, at 3:14 AM, Robert Dober wrote:

<snip>

Answered by Michael

Further, what if I have an instance var that is referencing some large data? Do I need to worry >about that being captured by the block or does it not matter because the block has already >captured self?

At first sight you need to worry if you pass the block to a method
outside of the receiver because that will create a new reference to
"self".
As long as you pass the block around in the same object I do not see an issue.

6

Robert Dober wrote in post #1013853:

<snip>

Answered by Michael

Further, what if I have an instance var that is referencing some large data? Do

I need to worry >about that being captured by the block or does it not
matter
because the block has already >captured self?

At first sight you need to worry if you pass the block to a method
outside of the receiver because that will create a new reference to
"self".
As long as you pass the block around in the same object I do not see an
issue.

Huh?

class Dog
  attr_accessor :val

  def initialize(x)
    @greeting = x
  end

  def make_proc
    return Proc.new { greet }
  end

  def greet
    puts @greeting
  end
end

d = Dog.new('lots of memory')
p = d.make_proc

d = nil
p.call

--output:--
lots of memory

···

--
Posted via http://www.ruby-forum.com/\.

7

<snip> Right?
That is what I meant, yes. Sorry I should have provided the code snippet ...
Cheers
Robert

···

On Sat, Jul 30, 2011 at 6:25 PM, Chuck Remes <cremes.devlist@mac.com> wrote:

On Jul 30, 2011, at 3:14 AM, Robert Dober wrote:

--
I'm not against types, but I don't know of any type systems that
aren't a complete pain, so I still like dynamic typing.
--
Alain Kay

8

May I ask what your point is?
R.

···

On Sun, Jul 31, 2011 at 9:27 AM, 7stud -- <bbxx789_05ss@yahoo.com> wrote:

Robert Dober wrote in post #1013853:

<snip>

--
I'm not against types, but I don't know of any type systems that
aren't a complete pain, so I still like dynamic typing.
--
Alain Kay

9

Robert Dober wrote in post #1013952:

···

On Sun, Jul 31, 2011 at 9:27 AM, 7stud -- <bbxx789_05ss@yahoo.com> > wrote:

Robert Dober wrote in post #1013853:

<snip>

May I ask what your point is?
R.

That what you said is wrong/doesn't make sense.

--
Posted via http://www.ruby-forum.com/\.