Nope, I can't agree with this because the ram goes up, the OS will kill it
eventually, and if I remove the guard the ram doesn't do this.
that makes perfect sense. with the gaurd only one thread at a time can be
initializing the huge array at once, it takes time, because of this the number
of threads grows quite large - thus the large amount of memory consumed as
they are added the the thread group. without the mutex the threads simply
race right through their work - creating the array and quickly exiting.
so, when the threads can be created and die quickly the maximum memory used by
the process simly never gets that big. with the mutex the maximum memory is
larger simply because the threads take longer to run.
run this on your system:
harp:~ > cat a.rb
pid = Process.pid
a =
new_array = lambda{|size| Array.new(size){ 42 }.map}
eat_memory = lambda{|n| n.times{ a << new_array[4242] }}
free_memory = lambda{a.clear and GC.start }
eat_memory[4242]
puts
puts "after malloc"
system "ps v #{ pid }"
free_memory
puts
puts "after free"
system "ps v #{ pid }"
harp:~ > ruby a.rb
after malloc
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31264 pts/10 R+ 0:24 0 595 27164 26364 15.6 ruby a.rb
after free
PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND
31264 pts/10 S+ 0:24 0 595 27164 26368 15.6 ruby a.rb
it shows the sort of behaviour i'm talking about
And where are you getting your information that free doesn't free
memory? I'd like to read that since all my years of C coding says that
is dead wrong.
http://groups.google.com/group/comp.unix.programmer/browse_frm/thread/44a5705312cc5df9/829197acdf6651ac?lnk=gst&q=memory+not+really+freed&rnum=4#829197acdf6651ac
http://groups.google.com/group/comp.unix.programmer/browse_frm/thread/23e7be26dd21434a/2f84b3dc080c7519?lnk=gst&q=memory+not+really+freed&rnum=9#2f84b3dc080c7519
the thing is that this has nothing to do with c and everything to do with os
Advanced Memory Allocation | Linux Journal
"When a process needs memory, some room is created by moving the upper bound of
the heap forward, using the brk() or sbrk() system calls. Because a system call
is expensive in terms of CPU usage, a better strategy is to call brk() to grab
a large chunk of memory and then split it as needed to get smaller chunks. This
is exactly what malloc() does. It aggregates a lot of smaller malloc() requests
into fewer large brk() calls. Doing so yields a significant performance
improvement. The malloc() call itself is much less expensive than brk(),
because it is a library call, not a system call. Symmetric behavior is adopted
when memory is freed by the process. Memory blocks are not immediately returned
to the system, which would require a new brk() call with a negative argument.
Instead, the C library aggregates them until a sufficiently large, contiguous
chunk can be freed at once.
For very large requests, malloc() uses the mmap() system call to find
addressable memory space. This process helps reduce the negative effects of
memory fragmentation when large blocks of memory are freed but locked by
smaller, more recently allocated blocks lying between them and the end of the
allocated space. In this case, in fact, had the block been allocated with
brk(), it would have remained unusable by the system even if the process freed
it."
Care to tell me how malloc/free would report 80M with Mutex but properly show
the ram go down when there is no-Mutex?
it's a sort of race
And why Linux would kill processes if the ram get too high? Why whole
VPS servers crash? I mean if this ram was just "fake" reporting (which
is very hard to believe) then why are all these things happening?
all memory is fake. that's why you can do this
harp:~ > free -b
total used free shared buffers cached
Mem: 1051602944 1032327168 19275776 0 145215488 563167232
^^^^^^^^
harp:~ > ruby -e' way_too_big = "42" * 19275776; p way_too_big.size '
38551552
regarding the crashes - there are still limits whic something is apparently
exceeding.
So please, point me at where in the specifications for malloc/free on
Linux it says that the memory reported will be high even though free and
malloc is called on 80M of ram slowly cycled out, and that linux will
still kill your process even though this ram is not really owned by the
process.
http://groups.google.com/group/comp.unix.programmer/search?group=comp.unix.programmer&q=memory+not+really+freed&qt_g=1&searchnow=Search+this+group
in any case, try running under electric fence, which will use a cleaner
malloc/free and show the 'real' behaviour. it really does seem fine.
regards.
-a
···
On Fri, 25 Aug 2006, Zed Shaw wrote:
--
to foster inner awareness, introspection, and reasoning is more efficient than
meditation and prayer.
- h.h. the 14th dalai lama
