Find memory leak in very complex Ruby app

ruby-talk
1

everyone!

It's nice to work with Ruby and write some code. But in past of this
week, i notice that we have some problem in our application. Memory
usage is growing like O(x*3) function.

Our application very complex, it is based on EventMachine and other
external libs. Even more, it is running under amd64 bit version of
FreeBSD using Ruby 1.8.7-p382

I'v tried to research by myself the way how find memory leak in our app.
I've found many tools and libs, but they doesn't work under
FreeBSD'64bit and I have no idea how step up to find leaks in huge ruby
application. It's OK, if you have few files with 200-300 lines of code,
but here you have around 30 files with average 200-300 line's of code.

I just realize, i need too much of time to find those leaks, doing
stupid actions: believe/research/assume that some of part of this code
is may be actually leaking and wrap some tracking code, like using
ruby-prof gem technice. But it's so painfully slow way, because as i
said we have too much of code.

So, my question is how to find memory leak in very complex Ruby app and
not put all my life into this work?

Thx in advance

···

--
Posted via http://www.ruby-forum.com/.

2

valgrind?

···

--
Posted via http://www.ruby-forum.com/.

3

I also recommend trying to run your code under JRuby and Rubinius. Both have very powerful tools for detecting and finding memory leaks, so those other runtimes may help shed some light on your problem.

cr

···

On Sep 14, 2011, at 11:31 AM, Tsyren Ochirov wrote:

everyone!

It's nice to work with Ruby and write some code. But in past of this
week, i notice that we have some problem in our application. Memory
usage is growing like O(x*3) function.

Our application very complex, it is based on EventMachine and other
external libs. Even more, it is running under amd64 bit version of
FreeBSD using Ruby 1.8.7-p382

I'v tried to research by myself the way how find memory leak in our app.
I've found many tools and libs, but they doesn't work under
FreeBSD'64bit and I have no idea how step up to find leaks in huge ruby
application. It's OK, if you have few files with 200-300 lines of code,
but here you have around 30 files with average 200-300 line's of code.

I just realize, i need too much of time to find those leaks, doing
stupid actions: believe/research/assume that some of part of this code
is may be actually leaking and wrap some tracking code, like using
ruby-prof gem technice. But it's so painfully slow way, because as i
said we have too much of code.

So, my question is how to find memory leak in very complex Ruby app and
not put all my life into this work?

4

In the absence-of/incompatibility of specialized tools or alternative
runtimes, an old-fashioned divide-and-conquer method works (regardless
of programming language/implementation):

1) loop your app in some way that provably leaks memory

2) Cut the code paths your app follows in half and have your loop
   follow one of them.

   a) First, split out the types of inputs your app receives from
      users (e.g. for an HTTP app, maybe all GET requests to a
      certain path).

   b) Neutralize code paths by editing them out. You can do this by
      placing early returns (and mocking/memoizing expected return
      values, if any) or commenting out code (including wrapping it
      with "if false; ...; end"

      Your app doesn't have to be correct while you're doing this,
      it just has to be able to loop successfully.

3) Repeat 1) with half your code active. If the new loop still leaks
   memory, repeat 2) to divide the code up into smaller chunks, and keep
   repeating until you find it. If it doesn't leak memory, follow the
   code path you didn't take.

A few notes:

* You can put the loop described in (1) in the app itself to speed
  things up (e.g. to avoid slow I/O).

* Reproducing the leak should get faster as the possible code paths to
  a leak become smaller.

* You may find multiple sources of leaks this way.

* You may need to go all the way down to C code (in Ruby itself or an
  extension) to find the bug.

* If you know your way around the app and already know /exactly/
  which code paths do not leak memory, you can neutralize those code
  paths to speed things up.

···

Tsyren Ochirov <tsyren.hey@gmail.com> wrote:

So, my question is how to find memory leak in very complex Ruby app and
not put all my life into this work?

5

And to expand on that:

The tips aren't Rails-specific, so don't be put off by the url. :slight_smile:

-Nick Klauer

···

On Wed, Sep 14, 2011 at 12:21, Roger Pack <rogerpack2005@gmail.com> wrote:

valgrind?

--
Posted via http://www.ruby-forum.com/\.

6

Roger Pack wrote in post #1021971:

valgrind?

Sorry that i'm sending you link, but

···

--
Posted via http://www.ruby-forum.com/\.

7

Chuck Remes wrote in post #1022009:

···

On Sep 14, 2011, at 11:31 AM, Tsyren Ochirov wrote:

I'v tried to research by myself the way how find memory leak in our app.

So, my question is how to find memory leak in very complex Ruby app and
not put all my life into this work?

I also recommend trying to run your code under JRuby and Rubinius. Both
have very powerful tools for detecting and finding memory leaks, so
those other runtimes may help shed some light on your problem.

cr

Chuck, thx to mention this way. I'd said that EM is bugging under JRuby,
and it's nice that you notice Rubinius. I'll try it!

--
Posted via http://www.ruby-forum.com/\.

8

Nick Klauer wrote in post #1021998:

And to expand on that:

The tips aren't Rails-specific, so don't be put off by the url. :slight_smile:

-Nick Klauer

Nice article, thx Nick!

···

--
Posted via http://www.ruby-forum.com/\.

9

Be sure you are running the latest Eventmachine from its master branch. I believe you can install it like so:

% gem install eventmachine --pre

If you are still running 0.12.10, then it's probable that Eventmachine is the source of your leak. Update to the latest and try again.

cr

···

On Sep 15, 2011, at 1:04 AM, Tsyren Ochirov wrote:

Chuck Remes wrote in post #1022009:

On Sep 14, 2011, at 11:31 AM, Tsyren Ochirov wrote:

I'v tried to research by myself the way how find memory leak in our app.

So, my question is how to find memory leak in very complex Ruby app and
not put all my life into this work?

I also recommend trying to run your code under JRuby and Rubinius. Both
have very powerful tools for detecting and finding memory leaks, so
those other runtimes may help shed some light on your problem.

cr

Chuck, thx to mention this way. I'd said that EM is bugging under JRuby,
and it's nice that you notice Rubinius. I'll try it!