I have a ruby script that reads data from a database, processes the data (long
texts) and saves the results back to the database. I works great and at good
speed.
The problem is that I ran the script on friday and left it until monday when I
saw some memory related errors in the console (Insufficient Memory
something...). I ran the script again and fired up the windows task manager
and then I saw that for every 10~15 records in the DB the script processes
the memory goes up 1MB and there are no signs of it going down.
I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.
Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??
I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.
any comments on how to find out where the memory is going are welcome.
I've had a problem like this using the flickr api and net/http. All it did was download images from Flickr, it ran pretty much 24/7 but some days it would just eat massive amounts of virtual ram, 900Mb plus.
I've wrapped the whole thing in a shell script now to get around the problem. Any idea how to track where the memory is going on a live system?
I have a ruby script that reads data from a database, processes the data (long
texts) and saves the results back to the database. I works great and at good
speed.
The problem is that I ran the script on friday and left it until monday when I
saw some memory related errors in the console (Insufficient Memory
something...). I ran the script again and fired up the windows task manager
and then I saw that for every 10~15 records in the DB the script processes
the memory goes up 1MB and there are no signs of it going down.
I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.
Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??
I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.
any comments on how to find out where the memory is going are welcome.
I already ran the Robert Klemme's code to see the object count of each class.
I ran the code every time my script finished processing each record of the
database and the numbers are like this:
These are the biggest entries but they do not increase all the time, they
fluctuate up and down. Strings goes up to 60,000+ objects and then drops
down back to 20,000+ objects. Then starts growing again. This happens with
all objects. Other classes are below 100 or are even 0.
I do not use symbols apart from those needed by ActiveRecord.
going to test now the MemoryProfiler of John Carter.
BTW, are these numbers normal??
thanks for the tips.
Horacio
Tuesday 13 December 2005 01:17、Gene Tani さんは書きました:
···
Horacio Sanson wrote:
> I have a ruby script that reads data from a database, processes the data
> (long texts) and saves the results back to the database. I works great
> and at good speed.
>
> The problem is that I ran the script on friday and left it until monday
> when I saw some memory related errors in the console (Insufficient Memory
> something...). I ran the script again and fired up the windows task
> manager and then I saw that for every 10~15 records in the DB the script
> processes the memory goes up 1MB and there are no signs of it going down.
>
> I started the script this morning and the PC was at 400MB ram usage, now
> is night and the PC is at 1.05GB ram usage.
>
> Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??
>
> I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.
>
> any comments on how to find out where the memory is going are welcome.
>
> regards,
> Horacio
I already ran the Robert Klemme's code to see the object count of each class.
I ran the code every time my script finished processing each record of the
database and the numbers are like this:
These are the biggest entries but they do not increase all the time, they
fluctuate up and down. Strings goes up to 60,000+ objects and then drops
down back to 20,000+ objects. Then starts growing again. This happens with
all objects. Other classes are below 100 or are even 0.
I do not use symbols apart from those needed by ActiveRecord.
going to test now the MemoryProfiler of John Carter.
BTW, are these numbers normal??
thanks for the tips.
Horacio
Tuesday 13 December 2005 01:17、Gene Tani さんは書きました:
> Horacio Sanson wrote:
> > I have a ruby script that reads data from a database, processes the data
> > (long texts) and saves the results back to the database. I works great
> > and at good speed.
> >
> > The problem is that I ran the script on friday and left it until monday
> > when I saw some memory related errors in the console (Insufficient Memory
> > something...). I ran the script again and fired up the windows task
> > manager and then I saw that for every 10~15 records in the DB the script
> > processes the memory goes up 1MB and there are no signs of it going down.
> >
> > I started the script this morning and the PC was at 400MB ram usage, now
> > is night and the PC is at 1.05GB ram usage.
> >
> > Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??
> >
> > I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.
> >
> > any comments on how to find out where the memory is going are welcome.
> >
> > regards,
> > Horacio
>
> a couple recent threads on digging into ruby'c GC:
> http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/4e77925c
>75949460
>
> and John Carter's thing to dump out your objects and (linux) processes
> http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/139987
i just found this, tagged by 1 del.icio.us user (thx whoeve you are!),
but it really talks about the same things.