# Ruby memory usage **URL:** \ **Category:** ruby-talk\ **Created:** [7 May 2009 18:01 UTC](https://rubytalk.org/t/ruby-memory-usage/53293 "2009-05-07T18:01:20Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [7 May 2009 18:01 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/1 "2009-05-07T18:01:20Z")
Hi Folks, I'm trying to figure out a memory usage issue with some long running ruby processes and was wondering if anyone can help me out. The background is that I have a set of long running ruby processes which are communicating with a larger system using an activeMQ server via STOMP. These processes are running for weeks at a time, most of the time just sitting there waiting for an incoming message. My team has noticed that these processes memory usage gradually grows over time. As a concrete example one of my processes starts life using around ~20MB of virtual memory, and 20 hours later is sitting at 382MB of virtual memory, 115MB resident. My first step in trying to figure this out was to use a slightly tweaked version of the memory profiler described here: [http://scottstuff.net/blog/2006/08/17/memory-leak-profiling-with-rails](http://scottstuff.net/blog/2006/08/17/memory-leak-profiling-with-rails). Essentially I'm just taking a snapshot of ObjectSpace every 10 seconds and dumping a top 20 count of how many instances there are of a class. Here are two snapshots, one take a few minutes after the process starts, and one ~20 hours later: ----------Wed May 06 14:03:33 -0700 2009---------- Top 20    23724: String     1121: Hash      776: Class      738: Regexp      578: Array      474: StatsBucket(id: integer, organization\_id: integer, se\_account\_id: integer, date: date)      474: TZInfo::TimezoneTransitionInfo      374: Proc      236: Module      154: Bignum      152: UnboundMethod      134: Range       62: Rational       57: Float       34: Gem::Version       28: ActiveRecord::ConnectionAdapters::MysqlColumn       27: Time       22: Gem::Version::Requirement       17: Mutex       16: Gem::Specification ----------Thu May 07 10:45:48 -0700 2009---------- Top 20    23848: String      834: Class      790: Bignum      783: Array      738: Regexp      506: Hash      474: TZInfo::TimezoneTransitionInfo      376: Proc      236: Module      159: StatsBucket(id: integer, organization\_id: integer, se\_account\_id: integer, date: date)      152: UnboundMethod      134: Range      129: Rational      107: Float      104: MatchData       54: Thread       46: Timeout::Error       45: Time       34: Gem::Version       32: ActiveRecord::ConnectionAdapters::MysqlColumn As you can see there's not that much of a change in the ObjectSpace, and yet the memory usage of this same process has blown up from 20MB to over 300MB of virtual memory. Can anyone give me any clues on what's going on here? Or any pointers as to what else I can in terms of measurements and instrumentation to get a better insight as to what's going here? Any help greatly appreciated! Cheers, Pete > **···** > > -- > Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/). ---
**Author:** ![Christopher\_Dicely](https://avatars.discourse-cdn.com/v4/letter/c/7c8e57/32.png) [@Christopher\_Dicely](https://rubytalk.org/u/Christopher_Dicely)\ **Post date:** [7 May 2009 19:02 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/2 "2009-05-07T19:02:05Z")
The first thing that jumps to mind is that the 54 instance of Thread in the later listing may be a sign of the problem; unless you really should have that many Threads hanging around, you may be doing something that is preventing used Thread objects from being garbage collected. If you are, for instance, launching a new thread when you get a message and they are hanging around after they aren't needed, that could explain the expanding memory usage. > **···** > > On Thu, May 7, 2009 at 11:01 AM, Pete Hodgson \ wrote: > > > Hi Folks, > > > > I'm trying to figure out a memory usage issue with some long running > > ruby processes and was wondering if anyone can help me out. > > > > The background is that I have a set of long running ruby processes which > > are communicating with a larger system using an activeMQ server via > > STOMP. These processes are running for weeks at a time, most of the time > > just sitting there waiting for an incoming message. > > > > My team has noticed that these processes memory usage gradually grows > > over time. As a concrete example one of my processes starts life using > > around ~20MB of virtual memory, and 20 hours later is sitting at 382MB > > of virtual memory, 115MB resident. > > > > My first step in trying to figure this out was to use a slightly tweaked > > version of the memory profiler described here: > > [http://scottstuff.net/blog/2006/08/17/memory-leak-profiling-with-rails\](http://scottstuff.net/blog/2006/08/17/memory-leak-profiling-with-rails%5C). > > Essentially I'm just taking a snapshot of ObjectSpace every 10 seconds > > and dumping a top 20 count of how many instances there are of a class. > > Here are two snapshots, one take a few minutes after the process starts, > > and one ~20 hours later: > > > > ----------Wed May 06 14:03:33 -0700 2009---------- > > Top 20 > > 23724: String > > 1121: Hash > > 776: Class > > 738: Regexp > > 578: Array > > 474: StatsBucket(id: integer, organization\_id: integer, > > se\_account\_id: integer, date: date) > > 474: TZInfo::TimezoneTransitionInfo > > 374: Proc > > 236: Module > > 154: Bignum > > 152: UnboundMethod > > 134: Range > > 62: Rational > > 57: Float > > 34: Gem::Version > > 28: ActiveRecord::ConnectionAdapters::MysqlColumn > > 27: Time > > 22: Gem::Version::Requirement > > 17: Mutex > > 16: Gem::Specification > > > > ----------Thu May 07 10:45:48 -0700 2009---------- > > Top 20 > > 23848: String > > 834: Class > > 790: Bignum > > 783: Array > > 738: Regexp > > 506: Hash > > 474: TZInfo::TimezoneTransitionInfo > > 376: Proc > > 236: Module > > 159: StatsBucket(id: integer, organization\_id: integer, > > se\_account\_id: integer, date: date) > > 152: UnboundMethod > > 134: Range > > 129: Rational > > 107: Float > > 104: MatchData > > 54: Thread > > 46: Timeout::Error > > 45: Time > > 34: Gem::Version > > 32: ActiveRecord::ConnectionAdapters::MysqlColumn > > > > As you can see there's not that much of a change in the ObjectSpace, and > > yet the memory usage of this same process has blown up from 20MB to over > > 300MB of virtual memory. > > > > Can anyone give me any clues on what's going on here? Or any pointers as > > to what else I can in terms of measurements and instrumentation to get a > > better insight as to what's going here? ---
**Author:** ![Tony\_Arcieri6](https://avatars.discourse-cdn.com/v4/letter/t/ecccb3/32.png) [@Tony\_Arcieri6](https://rubytalk.org/u/Tony_Arcieri6)\ **Post date:** [8 May 2009 03:57 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/3 "2009-05-08T03:57:52Z")
One thing to keep in mind is that MRI's garbage collector never returns memory to the system, and doesn't do a very good job in general. MRI has a tendency towards unbounded heap growth over time, especially in long-running processes. You might try JRuby if you don't have too many dependencies on C extensions. It uses the advanced garbage collection available in the JVM. > **···** > > On Thu, May 7, 2009 at 12:01 PM, Pete Hodgson \wrote: > > > My team has noticed that these processes memory usage gradually grows > > over time. As a concrete example one of my processes starts life using > > around ~20MB of virtual memory, and 20 hours later is sitting at 382MB > > of virtual memory, 115MB resident. > > -- > Tony Arcieri > medioh.com ---
**Author:** ![Roger\_Pack4](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@Roger\_Pack4](https://rubytalk.org/u/Roger_Pack4)\ **Post date:** [8 May 2009 12:27 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/4 "2009-05-08T12:27:07Z")
Pete Hodgson wrote: > Hi Folks, > > I'm trying to figure out a memory usage issue with some long running > ruby processes and was wondering if anyone can help me out. If it's poor GC you could try out the MBARI patches. I had a rails app that grew by 8MB /request \*every request\* until I switched to MBARI--now it works like a charm. Good luck! -=r > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [7 May 2009 22:19 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/5 "2009-05-07T22:19:40Z")
Christopher Dicely wrote: > **···** > > > On Thu, May 7, 2009 at 11:01 AM, Pete Hodgson \ \> wrote: > > > > > My team has noticed that these processes memory usage gradually grows > > > and one ~20 hours later: > > >     474: TZInfo::TimezoneTransitionInfo > > >      22: Gem::Version::Requirement > > >     738: Regexp > > >     104: MatchData > > > 300MB of virtual memory. > > > > > > Can anyone give me any clues on what's going on here? Or any pointers as > > > to what else I can in terms of measurements and instrumentation to get a > > > better insight as to what's going here? > > > > The first thing that jumps to mind is that the 54 instance of Thread > > in the later listing may be a sign of the problem; unless you really > > should have that many Threads hanging around, you may be doing > > something that is preventing used Thread objects from being garbage > > collected. If you are, for instance, launching a new thread when you > > get a message and they are hanging around after they aren't needed, > > that could explain the expanding memory usage. > > Hmm, you're right that is strange. I can't think of anywhere where I am > spawning threads after initial start up, so I'll look into that more and > see where that leads. Thanks Christopher! > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\ **Post date:** [8 May 2009 09:11 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/6 "2009-05-08T09:11:38Z")
I believe this has changed with 1.9.1 - at least memory usage seems to not grow as much as with 1.8.\*. Btw, you can easily construct a memory leak, which cannot necessarily be detected by counting objects: ruby19 -e 's="."\*1024;t="";loop do t \<\< s end' I.e., you can grow any String which is not frozen to arbitrary lengths. Btw, Pete, you might get more insight into your application if you dump deltas of object counts. IIRC we had a sample here in the group but I cannot find it at the moment. Kind regards robert > **···** > > 2009/5/8 Tony Arcieri \: > > > On Thu, May 7, 2009 at 12:01 PM, Pete Hodgson \wrote: > > > > > My team has noticed that these processes memory usage gradually grows > > > over time. As a concrete example one of my processes starts life using > > > around ~20MB of virtual memory, and 20 hours later is sitting at 382MB > > > of virtual memory, 115MB resident. > > > > One thing to keep in mind is that MRI's garbage collector never returns > > memory to the system, and doesn't do a very good job in general. MRI has a > > tendency towards unbounded heap growth over time, especially in long-running > > processes. > > -- > remember.guy do |as, often| as.you\_can - without end > [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/) ---
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [8 May 2009 20:49 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/7 "2009-05-08T20:49:04Z")
Thanks so much for all the input folks, it's really really appreciated. ruby -v gives: ruby 1.8.6 (2008-06-20 patchlevel 230) [i386-linux] I did some additional tweaks to the memory profiler, and it now dumps a yaml document with object space details every 10 seconds. I did some analysis on discovered that: - Virtual memory grows in a straight line serially over time - Resident memory grows up to a point, then start fluctuating. I'm presuming that's the kernel starting to reclaim resident memory from the process as it starts to run out of physical memory. - Object counts stay almost entirely flat over time. At this point I wonder whether the issue is with the MRI GC, as Tony mentioned. I'm a little leary of upgrading to 1.9 just in order to fix this, but I might give it a try just to see what the memory usage over time looks like compared to 1.8.6. > **···** > > -- > Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/). ---
**Author:** ![Nobuyoshi\_Nakada1](https://avatars.discourse-cdn.com/v4/letter/n/9f8e36/32.png) [@Nobuyoshi\_Nakada1](https://rubytalk.org/u/Nobuyoshi_Nakada1)\ **Post date:** [9 May 2009 00:20 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/8 "2009-05-09T00:20:37Z")
Hi, At Fri, 8 May 2009 12:57:52 +0900, Tony Arcieri wrote in [ruby-talk:336091]: > One thing to keep in mind is that MRI's garbage collector never returns > memory to the system, and doesn't do a very good job in general. MRI has a > tendency towards unbounded heap growth over time, especially in long-running > processes. You know only pretty old versions. > **···** > > -- > Nobu Nakada ---
**Author:** ![Xeno\_Campanoli1](https://avatars.discourse-cdn.com/v4/letter/x/85f322/32.png) [@Xeno\_Campanoli1](https://rubytalk.org/u/Xeno_Campanoli1)\ **Post date:** [7 May 2009 23:11 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/9 "2009-05-07T23:11:40Z")
Pete Hodgson wrote: > Christopher Dicely wrote: > > > > My team has noticed that these processes memory usage gradually grows For what it's worth, I have a pretty complex web app with multiple dimensional arrays referencing nested classes and it runs really slow when we have less than a gig of memory, which never really made sense to us for the level of stuff it was doing. I may just need to spend a lot more time cleaning up my code, but seems to me Ruby can eat a bit more memory than one might expect it should. xc > **···** > > > > On Thu, May 7, 2009 at 11:01 AM, Pete Hodgson \ \>\> wrote: > > > > > and one ~20 hours later: > > > >     474: TZInfo::TimezoneTransitionInfo > > > >      22: Gem::Version::Requirement > > > >     738: Regexp > > > >     104: MatchData > > > > 300MB of virtual memory. > > > > > > > > Can anyone give me any clues on what's going on here? Or any pointers as > > > > to what else I can in terms of measurements and instrumentation to get a > > > > better insight as to what's going here? > > > > > > The first thing that jumps to mind is that the 54 instance of Thread > > > in the later listing may be a sign of the problem; unless you really > > > should have that many Threads hanging around, you may be doing > > > something that is preventing used Thread objects from being garbage > > > collected. If you are, for instance, launching a new thread when you > > > get a message and they are hanging around after they aren't needed, > > > that could explain the expanding memory usage. > > > > Hmm, you're right that is strange. I can't think of anywhere where I am spawning threads after initial start up, so I'll look into that more and see where that leads. Thanks Christopher! > > -- > "I told the Republicans that when they stopped lying about me I'd stop > telling the truth about them" -- Harry Truman. ---
**Author:** ![Bill\_Kelly](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@Bill\_Kelly](https://rubytalk.org/u/Bill_Kelly)\ **Post date:** [7 May 2009 23:32 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/10 "2009-05-07T23:32:23Z")
> Christopher Dicely wrote: > > > The first thing that jumps to mind is that the 54 instance of Thread > > in the later listing may be a sign of the problem; unless you really > > should have that many Threads hanging around, you may be doing > > something that is preventing used Thread objects from being garbage > > collected. If you are, for instance, launching a new thread when you > > get a message and they are hanging around after they aren't needed, > > that could explain the expanding memory usage. > > Hmm, you're right that is strange. I can't think of anywhere where I am spawning threads after initial start up, so I'll look into that more and see where that leads. Thanks Christopher! Also, what's your ruby -v ? Some versions of 1.8.6 have leaked memory, for instance. Regards, Bill > **···** > > From: "Pete Hodgson" \ ---
**Author:** ![Joel\_VanderWerf1](https://avatars.discourse-cdn.com/v4/letter/j/94ad74/32.png) [@Joel\_VanderWerf1](https://rubytalk.org/u/Joel_VanderWerf1)\ **Post date:** [8 May 2009 16:24 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/11 "2009-05-08T16:24:01Z")
Robert Klemme wrote: > Btw, you can easily construct a memory leak, which cannot necessarily > be detected by counting objects: > > ruby19 -e 's="."\*1024;t="";loop do t \<\< s end' > > I.e., you can grow any String which is not frozen to arbitrary lengths. That not really a leak, though, since t is referenced... Not that this situation isn't a potential problem, but we should make sure everyone understands that the blame for it rests squarely on the programmer and not on ruby's GC. > **···** > > -- >        vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407 ---
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [8 May 2009 21:34 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/12 "2009-05-08T21:34:02Z")
Pete Hodgson wrote: > Thanks so much for all the input folks, it's really really appreciated. > > ruby -v gives: > ruby 1.8.6 (2008-06-20 patchlevel 230) [i386-linux] > > I did some additional tweaks to the memory profiler, and it now dumps a > yaml document with object space details every 10 seconds. I did some > analysis on discovered that: > > - Virtual memory grows in a straight line serially over time > - Resident memory grows up to a point, then start fluctuating. I'm > presuming that's the kernel starting to reclaim resident memory from the > process as it starts to run out of physical memory. > - Object counts stay almost entirely flat over time. > > At this point I wonder whether the issue is with the MRI GC, as Tony > mentioned. I'm a little leary of upgrading to 1.9 just in order to fix > this, but I might give it a try just to see what the memory usage over > time looks like compared to 1.8.6. I charted out memory usage over time and object space usage over time to illustrate what I'm seeing: [http://dl.getdropbox.com/u/796485/mercury\_mem\_usage.jpg](http://dl.getdropbox.com/u/796485/mercury_mem_usage.jpg) [http://dl.getdropbox.com/u/796485/mercury\_object\_counts.jpg](http://dl.getdropbox.com/u/796485/mercury_object_counts.jpg) > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Tony\_Arcieri6](https://avatars.discourse-cdn.com/v4/letter/t/ecccb3/32.png) [@Tony\_Arcieri6](https://rubytalk.org/u/Tony_Arcieri6)\ **Post date:** [8 May 2009 21:58 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/13 "2009-05-08T21:58:50Z")
> Thanks so much for all the input folks, it's really really appreciated. > > ruby -v gives: > ruby 1.8.6 (2008-06-20 patchlevel 230) [i386-linux] > > I did some additional tweaks to the memory profiler, and it now dumps a > yaml document with object space details every 10 seconds. I think memory profiling using ObjectSpace is fundamentally flawed (a few years ago I wrote an ObjectSpace-powered memory profiler and had many issues with it) You might check out bleak\_house instead: > **[Evan Weaver](https://evan.vc)** > > I'm an engineer, founder, and angel investor in software infrastructure, developer tools, social media, ecommerce, and AI. > At this point I wonder whether the issue is with the MRI GC, as Tony > mentioned. I'm a little leary of upgrading to 1.9 just in order to fix > this, but I might give it a try just to see what the memory usage over > time looks like compared to 1.8.6. With JRuby you can have 1.8 compatibility plus your selection of all the pluggable garbage collectors available in the JVM. > **···** > > On Fri, May 8, 2009 at 2:49 PM, Pete Hodgson \ wrote: > > -- > Tony Arcieri > medioh.com ---
**Author:** ![Bill\_Kelly](https://avatars.discourse-cdn.com/v4/letter/b/cc9497/32.png) [@Bill\_Kelly](https://rubytalk.org/u/Bill_Kelly)\ **Post date:** [8 May 2009 23:51 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/14 "2009-05-08T23:51:36Z")
> ruby -v gives: > ruby 1.8.6 (2008-06-20 patchlevel 230) [i386-linux] patchlevel 230 was one of the bad ones. It leaks like a sieve. See [ruby-core] thread: [Ruby 1.8 - Bug #216] Memory leaks in 1.8.6p230 and p238 [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17571](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17571) It affected a whole range of patchlevels: [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17596](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17596) Also, a leak of similar severity was introduced later on, starting in p296 lasting at least through p355: [ruby-core:22582] Re: [Bug #1223] Memory leak reintroduced in 1.8.6 branch? [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/22852](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/22852) I'd recommend trying the recent official 1.8.6 release, p368 [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/332578](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/332578) I have not personally tried p368 on my long-running servers, as I had already found a stable version, p279. So I have not had a need to upgrade the version. I can say that p279 is very stable in its memory profile. I'm guessing the official p368 release probably is too. Hope this helps, Bill > **···** > > From: "Pete Hodgson" \ ---
**Author:** ![Robert\_K1](https://yyz1.discourse-cdn.com/flex029/user_avatar/rubytalk.org/robert_k1/32/1830_2.png) [@Robert\_K1](https://rubytalk.org/u/Robert_K1)\ **Post date:** [8 May 2009 16:30 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/15 "2009-05-08T16:30:06Z")
> Robert Klemme wrote: > > > Btw, you can easily construct a memory leak, which cannot necessarily > > be detected by counting objects: > > > > ruby19 -e 's="."\*1024;t="";loop do t \<\< s end' > > > > I.e., you can grow any String which is not frozen to arbitrary lengths. > > That not really a leak, though, since t is referenced... Well, AFAIK the definition of "memory leak" in GC languages is that of objects kept longer than needed. So if someone creates a program which simply appends text to a String over and over again where creating new short strings would be sufficient he has created a memory leak IMHO. My main point though was to demonstrate that memory usage and object count do not necessarily correlate. > Not that this situation isn't a potential problem, but we should make sure everyone understands that the blame for it rests squarely on the programmer and not on ruby's GC. Yep! Kind regards   robert > **···** > > On 08.05.2009 18:24, Joel VanderWerf wrote: > > -- > remember.guy do |as, often| as.you\_can - without end > [http://blog.rubybestpractices.com/](http://blog.rubybestpractices.com/) ---
**Author:** ![Roger\_Pack4](https://avatars.discourse-cdn.com/v4/letter/r/bb73d2/32.png) [@Roger\_Pack4](https://rubytalk.org/u/Roger_Pack4)\ **Post date:** [8 May 2009 17:04 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/16 "2009-05-08T17:04:09Z")
> > ruby19 -e 's="."\*1024;t="";loop do t \<\< s end' > > > > I.e., you can grow any String which is not frozen to arbitrary lengths. > > That not really a leak, though, since t is referenced... > > Not that this situation isn't a potential problem, but we should make > sure everyone understands that the blame for it rests squarely on the > programmer and not on ruby's GC. In this particular case the problem definitely lies with the programmer. However, in some other instances ruby's GC really does stink (but hopefully the MBARI patches overcome its deficiencies, and also Ruby 1.9 should have fewer problems since its threads use distinct thread memory spaces so can't share ghost references). Cheers! -=r > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Fleck\_Jean-Julien](https://avatars.discourse-cdn.com/v4/letter/f/65b543/32.png) [@Fleck\_Jean-Julien](https://rubytalk.org/u/Fleck_Jean-Julien)\ **Post date:** [8 May 2009 21:55 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/17 "2009-05-08T21:55:46Z")
Hello, > I charted out memory usage over time and object space usage over time to > illustrate what I'm seeing: > > [http://dl.getdropbox.com/u/796485/mercury\_mem\_usage.jpg](http://dl.getdropbox.com/u/796485/mercury_mem_usage.jpg) > [http://dl.getdropbox.com/u/796485/mercury\_object\_counts.jpg](http://dl.getdropbox.com/u/796485/mercury_object_counts.jpg) Which type of object is the red curve (between 500 and 1000) that makes one jump up when memory start raising and begins to behave erraticaly approximately at the same time resident memory does ? Cheers, JJ ---
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [8 May 2009 22:49 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/18 "2009-05-08T22:49:12Z")
Tony Arcieri wrote: > > Thanks so much for all the input folks, it's really really appreciated. > > > > ruby -v gives: > > ruby 1.8.6 (2008-06-20 patchlevel 230) [i386-linux] > > > > I did some additional tweaks to the memory profiler, and it now dumps a > > yaml document with object space details every 10 seconds. > > I think memory profiling using ObjectSpace is fundamentally flawed (a > few > years ago I wrote an ObjectSpace-powered memory profiler and had many > issues > with it) > > You might check out bleak\_house instead: > > [Evan Weaver](http://blog.evanweaver.com/files/doc/fauna/bleak_house/files/README.html) I'm thinking that is probably gonna have to be the next step. I had a bad experience trying to set it up in the past and now am a bit gun shy, but I'm running out of options here... > > At this point I wonder whether the issue is with the MRI GC, as Tony > > mentioned. I'm a little leary of upgrading to 1.9 just in order to fix > > this, but I might give it a try just to see what the memory usage over > > time looks like compared to 1.8.6. > > With JRuby you can have 1.8 compatibility plus your selection of all the > pluggable garbage collectors available in the JVM. Thanks, I'll give JRuby a run on a VM and see how it performs. I'm guessing that bleak\_house and JRuby don't play together? > **···** > > > On Fri, May 8, 2009 at 2:49 PM, Pete Hodgson \ \> wrote: > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [9 May 2009 21:21 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/19 "2009-05-09T21:21:11Z")
Bill Kelly wrote: > From: "Pete Hodgson" \ > > > ruby -v gives: > > ruby 1.8.6 (2008-06-20 patchlevel 230) [i386-linux] > > patchlevel 230 was one of the bad ones. It leaks like a sieve. > > See [ruby-core] thread: > > [Ruby 1.8 - Bug #216] Memory leaks in 1.8.6p230 and p238 > [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17571](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17571) > > It affected a whole range of patchlevels: > [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17596](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/17596) > > Also, a leak of similar severity was introduced later on, starting > in p296 lasting at least through p355: > > [ruby-core:22582] Re: [Bug #1223] Memory leak reintroduced in 1.8.6 > branch? > [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/22852](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/22852) > > I'd recommend trying the recent official 1.8.6 release, p368 > [http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/332578](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/332578) > > I have not personally tried p368 on my long-running servers, as > I had already found a stable version, p279. So I have not had a > need to upgrade the version. I can say that p279 is very stable > in its memory profile. I'm guessing the official p368 release > probably is too. > > Hope this helps, > > Bill Great info Bill, thanks so much. I'll try building 1.8.6-p368 and see if the same component exhibits different behaviour running under that build. Assuming that helps, does anyone have any pointers on putting a non-standard ruby rpm onto a centOS box? My company use centos in production and I'm betting that my Ops guys will want an rpm, official or otherwise. > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). ---
**Author:** ![Pete\_Hodgson1](https://avatars.discourse-cdn.com/v4/letter/p/e99b99/32.png) [@Pete\_Hodgson1](https://rubytalk.org/u/Pete_Hodgson1)\ **Post date:** [8 May 2009 22:46 UTC](https://rubytalk.org/t/ruby-memory-usage/53293/20 "2009-05-08T22:46:50Z")
Fleck Jean-Julien wrote: > Hello, > > > I charted out memory usage over time and object space usage over time to > > illustrate what I'm seeing: > > > > [http://dl.getdropbox.com/u/796485/mercury\_mem\_usage.jpg](http://dl.getdropbox.com/u/796485/mercury_mem_usage.jpg) > > [http://dl.getdropbox.com/u/796485/mercury\_object\_counts.jpg](http://dl.getdropbox.com/u/796485/mercury_object_counts.jpg) > > Which type of object is the red curve (between 500 and 1000) that > makes one jump up when memory start raising and begins to behave > erraticaly approximately at the same time resident memory does ? > > Cheers, > > JJ instances of Array. Not sure why it starts behaving like that at the same time residential memory does. > **···** > > -- > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C). [Next page](https://rubytalk.org/t/ruby-memory-usage/53293.md?page=2)