re: my previous post, looking for brk and sbrk will be informative also
Gary Wright
···
On Aug 28, 2006, at 7:26 PM, Eric Hodel wrote:
brk and sbrk are still used, for example, phkmalloc:
I posted this earlier, but gmane seems to have snarfed it. Apologies if it
shows up twice.
gwtmp01@mac.com writes:
Run your code under a process trace program such as ktrace (Mac OS X). You
should be able to easily grep the resulting dump for system calls that
*actually* return memory to the OS. Just because free is called doesn't mean
that the memory allocator has actually notified the kernel that it no longer
needs that huge hunk of memory.
I posted some graphs in an earlier message:
http://article.gmane.org/gmane.comp.lang.ruby.general/168484
But I now have some prettier versions from 2-hour runs (apologies for the image
sizes):
http://llasram.wuli.nu/mutex.png
http://llasram.wuli.nu/sync.png
In these graphs the red line represents the amount of memory Ruby is allocating
from the heap. The blue-green regions represent where those allocations are
coming from in byte offsets from the base of the heap. These graphs no longer
have the brk() line indicating the end of the heap because (on Linux anyway) it
exactly tracks the highest allocated heap memory address.
This visually demonstrates that the heap is becoming highly fragmented with
both scripts, albeit worse with 'mutex.rb'.
I'm guessing you'll look for calls to munmap or something similar.
At least on Linux, the allocation pattern used by these scripts never causes
the heap implementation to get system memory via m/unmap() (as you can see
visually in my graphs by how the amount ruby allocates using malloc() never
goes beyond the size of the heap, even during peak allocation periods).
-Marshall
Right, so brk/sbrk, but in the environment you are testing, the process never
actually gives memory back to the kernel. So even if the memory isn't used
(i.e. it isn't part of the working set of the process), from the point of view
of the kernel it is allocated to the process and so occupies various tables and
swap/paging space within the kernel.
Gary Wright
···
On Aug 29, 2006, at 11:13 PM, Marshall T. Vandegrift wrote:
At least on Linux, the allocation pattern used by these scripts never causes
the heap implementation to get system memory via m/unmap() (as you can see
visually in my graphs by how the amount ruby allocates using malloc() never
goes beyond the size of the heap, even during peak allocation periods).