Hi
1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB
1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MB
Mem-consumation is growing with each call.
Why is the memory not freed? (bad if having much data, like long
strings, returned!
Opti
Hi,
There are a lot of different factors at play here like:
- where did the 130 mb come from, did you measure this in GC.stat or is the
OS memory?
- what os are you running;
- which ruby are you running (MRI / jruby etc);
- and, in case of the MRI, which memory allocator are you using (malloc /
jemalloc).
In general (this applies to not only ruby), not all freed memory is
necessarily returned to the OS (by a memory allocator) since this is an
expensive operation. So from an OS point it might be "taken", but it can
still be used by a memory allocator to hand out to, in this case, Ruby.
There are several good explanations out there how Ruby GC works under the
hood. For instance: https://scoutapm.com/blog/ruby-garbage-collection is a
nice read.
And if you want to dive deeper: Aaron Patterson also has some excellent
video's on the work he and others have been doing on building a compacting
gc. For instance: https://www.youtube.com/watch?v=H8iWLoarTZc
Hope this helps.
Lars
On Wed, May 25, 2022 at 10:03 AM Die Optimisten <inform@die-optimisten.net> wrote:
Hi
1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MBMem-consumation is growing with each call.
Why is the memory not freed? (bad if having much data, like long
strings, returned!Opti
Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>
Hi,
as you can see, each time you add a line x.times {...}, more memory is
CONSUMED (additionally allocated from the OS), meaning that it's
allocating more memory from the OS every time (Linux, using MRI). - it's
not the question of giving memory back to the OS.
So if using longer strings (or doing it more often), you'll go out of
memory, which is not what you want. So the used memory is not
(correctly) freed by GC (for further usage by the running Ruby-process).
Opti
Am 25.05.22 um 17:36 schrieb Lars Vonk:
Hi,
There are a lot of different factors at play here like:
- where did the 130 mb come from, did you measure this in GC.stat or
is the OS memory?
- what os are you running;
- which ruby are you running (MRI / jruby etc);
- and, in case of the MRI, which memory allocator are you using
(malloc / jemalloc).In general (this applies to not only ruby), not all freed memory is
necessarily returned to the OS (by a memory allocator) since this is
an expensive operation. So from an OS point it might be "taken", but
it can still be used by a memory allocator to hand out to, in this
case, Ruby.There are several good explanations out there how Ruby GC works under
the hood. For instance:
https://scoutapm.com/blog/ruby-garbage-collection is a nice read.And if you want to dive deeper: Aaron Patterson also has some
excellent video's on the work he and others have been doing on
building a compacting gc. For instance:
https://www.youtube.com/watch?v=H8iWLoarTZcHope this helps.
Lars
On Wed, May 25, 2022 at 10:03 AM Die Optimisten > <inform@die-optimisten.net <mailto:inform@die-optimisten.net>> wrote:
Hi
1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MBMem-consumation is growing with each call.
Why is the memory not freed? (bad if having much data, like long
strings, returned!)Opti
GC is much more complex than you assume. You cannot expect a simple
one-to-one relationship between allocation and de-allocation over short
term time.
-gf-
On Wed, May 25, 2022 at 12:39 PM Die Optimisten <inform@die-optimisten.net> wrote:
Hi,
as you can see, each time you add a line x.times {...}, more memory is
CONSUMED (additionally allocated from the OS), meaning that it's allocating
more memory from the OS every time (Linux, using MRI). - it's not the
question of giving memory back to the OS.
So if using longer strings (or doing it more often), you'll go out of
memory, which is not what you want. So the used memory is not (correctly)
freed by GC (for further usage by the running Ruby-process).Opti
Am 25.05.22 um 17:36 schrieb Lars Vonk:
Hi,
There are a lot of different factors at play here like:
- where did the 130 mb come from, did you measure this in GC.stat or is
the OS memory?
- what os are you running;
- which ruby are you running (MRI / jruby etc);
- and, in case of the MRI, which memory allocator are you using (malloc /
jemalloc).In general (this applies to not only ruby), not all freed memory is
necessarily returned to the OS (by a memory allocator) since this is an
expensive operation. So from an OS point it might be "taken", but it can
still be used by a memory allocator to hand out to, in this case, Ruby.There are several good explanations out there how Ruby GC works under the
hood. For instance: https://scoutapm.com/blog/ruby-garbage-collection is
a nice read.And if you want to dive deeper: Aaron Patterson also has some excellent
video's on the work he and others have been doing on building a compacting
gc. For instance: https://www.youtube.com/watch?v=H8iWLoarTZcHope this helps.
Lars
On Wed, May 25, 2022 at 10:03 AM Die Optimisten <inform@die-optimisten.net> > wrote:
Hi
1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 130MB1000000.times.map{|i| v=['abc'] }; GC.start
Memory-Usage: 133MBMem-consumation is growing with each call.
Why is the memory not freed? (bad if having much data, like long strings,
returned!)Opti
Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>