memoryleak - bug

Hi

1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB

1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MB

Mem-consumation is growing with each call.

Why is the memory not freed? (bad if having much data, like long
strings, returned!

Opti

Hi,

There are a lot of different factors at play here like:

- where did the 130 mb come from, did you measure this in GC.stat or is the
OS memory?
- what os are you running;
- which ruby are you running (MRI / jruby etc);
- and, in case of the MRI, which memory allocator are you using (malloc /
jemalloc).

In general (this applies to not only ruby), not all freed memory is
necessarily returned to the OS (by a memory allocator) since this is an
expensive operation. So from an OS point it might be "taken", but it can
still be used by a memory allocator to hand out to, in this case, Ruby.

There are several good explanations out there how Ruby GC works under the
hood. For instance: Ruby Garbage Collection | Memory Management | Scout APM Blog is a
nice read.

And if you want to dive deeper: Aaron Patterson also has some excellent
video's on the work he and others have been doing on building a compacting
gc. For instance: Compacting GC in Ruby 2.7 - Aaron Patterson - YouTube

Hope this helps.

Lars

···

On Wed, May 25, 2022 at 10:03 AM Die Optimisten <inform@die-optimisten.net> wrote:

Hi

1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB

1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MB

Mem-consumation is growing with each call.

Why is the memory not freed? (bad if having much data, like long
strings, returned!

Opti

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<ruby-talk list: member options login page>

Hi,
as you can see, each time you add a line x.times {...}, more memory is
CONSUMED (additionally allocated from the OS), meaning that it's
allocating more memory from the OS every time (Linux, using MRI). - it's
not the question of giving memory back to the OS.
So if using longer strings (or doing it more often), you'll go out of
memory, which is not what you want. So the used memory is not
(correctly) freed by GC (for further usage by the running Ruby-process).

Opti

···

Am 25.05.22 um 17:36 schrieb Lars Vonk:

Hi,

There are a lot of different factors at play here like:

- where did the 130 mb come from, did you measure this in GC.stat or
is the OS memory?
- what os are you running;
- which ruby are you running (MRI / jruby etc);
- and, in case of the MRI, which memory allocator are you using
(malloc / jemalloc).

In general (this applies to not only ruby), not all freed memory is
necessarily returned to the OS (by a memory allocator) since this is
an expensive operation. So from an OS point it might be "taken", but
it can still be used by a memory allocator to hand out to, in this
case, Ruby.

There are several good explanations out there how Ruby GC works under
the hood. For instance:
Ruby Garbage Collection | Memory Management | Scout APM Blog is a nice read.

And if you want to dive deeper: Aaron Patterson also has some
excellent video's on the work he and others have been doing on
building a compacting gc. For instance:
Compacting GC in Ruby 2.7 - Aaron Patterson - YouTube

Hope this helps.

Lars

On Wed, May 25, 2022 at 10:03 AM Die Optimisten > <inform@die-optimisten.net <mailto:inform@die-optimisten.net>> wrote:

    Hi

    1000000.times.map{|i| v=['abc'] }; GC.start
    Memory-Usage: 130MB

    1000000.times.map{|i| v=['abc'] }; GC.start
    Memory-Usage: 133MB

    Mem-consumation is growing with each call.

    Why is the memory not freed? (bad if having much data, like long
    strings, returned!)

    Opti

GC is much more complex than you assume. You cannot expect a simple
one-to-one relationship between allocation and de-allocation over short
term time.

-gf-

···

On Wed, May 25, 2022 at 12:39 PM Die Optimisten <inform@die-optimisten.net> wrote:

Hi,
as you can see, each time you add a line x.times {...}, more memory is
CONSUMED (additionally allocated from the OS), meaning that it's allocating
more memory from the OS every time (Linux, using MRI). - it's not the
question of giving memory back to the OS.
So if using longer strings (or doing it more often), you'll go out of
memory, which is not what you want. So the used memory is not (correctly)
freed by GC (for further usage by the running Ruby-process).

Opti

Am 25.05.22 um 17:36 schrieb Lars Vonk:

Hi,

There are a lot of different factors at play here like:

- where did the 130 mb come from, did you measure this in GC.stat or is
the OS memory?
- what os are you running;
- which ruby are you running (MRI / jruby etc);
- and, in case of the MRI, which memory allocator are you using (malloc /
jemalloc).

In general (this applies to not only ruby), not all freed memory is
necessarily returned to the OS (by a memory allocator) since this is an
expensive operation. So from an OS point it might be "taken", but it can
still be used by a memory allocator to hand out to, in this case, Ruby.

There are several good explanations out there how Ruby GC works under the
hood. For instance: Ruby Garbage Collection | Memory Management | Scout APM Blog is
a nice read.

And if you want to dive deeper: Aaron Patterson also has some excellent
video's on the work he and others have been doing on building a compacting
gc. For instance: Compacting GC in Ruby 2.7 - Aaron Patterson - YouTube

Hope this helps.

Lars

On Wed, May 25, 2022 at 10:03 AM Die Optimisten <inform@die-optimisten.net> > wrote:

Hi

1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB

1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MB

Mem-consumation is growing with each call.

Why is the memory not freed? (bad if having much data, like long strings,
returned!)

Opti

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<ruby-talk list: member options login page>