Manual Memory Management and Automatic Garbage Collection

Hello all

I am trying to do a research on Manual Memory Management and Automatic
Garbage Collection on Ruby.So i need to play with Garbage Collector code
of Ruby.I am using Ruby 1.8.6

I am also trying to figure out in Ruby when the Garbage Collector starts
working.But didn't got any clue.

I read somewhere in wiki about some systems that uses this design of
Memory Allocation.

Can any one help me in some of my queries:-

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

After I get my first step done I hope I will have many questions to
share with you all.

Regards...Eagerly waiting for a reply at the earliest

Tridib

···

--
Posted via http://www.ruby-forum.com/.

Have you tried sprinkling some printf statements throughout gc.c
yourself yet? If the diagnostic instrumentation for gc.c is hard to
find or does not exist, adding your own can at least help you make some
progress.

To start, you could add a printf with the function name at the beginning
of each function in gc.c. Then run your build of ruby with your test
scripts to trigger garbage collection and see what gets printed. That
should give you a lead on where to add more printf calls to gc.c in
order to gather more detailed information.

-Jeremy

···

On 12/6/2010 8:41 AM, Tridib Bandopadhyay wrote:

Hello all

I am trying to do a research on Manual Memory Management and Automatic
Garbage Collection on Ruby.So i need to play with Garbage Collector code
of Ruby.I am using Ruby 1.8.6

I am also trying to figure out in Ruby when the Garbage Collector starts
working.But didn't got any clue.

I read somewhere in wiki about some systems that uses this design of
Memory Allocation.

Can any one help me in some of my queries:-

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

After I get my first step done I hope I will have many questions to
share with you all.

Regards...Eagerly waiting for a reply at the earliest

I am trying to do a research on Manual Memory Management and Automatic
Garbage Collection on Ruby.So i need to play with Garbage Collector code
of Ruby.I am using Ruby 1.8.6

I am also trying to figure out in Ruby when the Garbage Collector starts
working.But didn't got any clue.

There were some threads here recently about GC. What specific
questions were left unanswered?

I read somewhere in wiki about some systems that uses this design of
Memory Allocation.

Can any one help me in some of my queries:-

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

You know CPP (C preprocessor) defines and macros, do you? Please look
for "GC_NOTIFY".

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

I haven't looked closely but I would start at the end of the file and
walk my way through reading from rb_gc_start(). You should hit all
main structures pretty soon.

After I get my first step done I hope I will have many questions to
share with you all.

OK, let's hear them.

Regards...Eagerly waiting for a reply at the earliest

Please don't push it. It won't increase your chances of getting answers anyway.
http://www.catb.org/~esr/faqs/smart-questions.html#urgent

Cheers

robert

···

On Mon, Dec 6, 2010 at 3:41 PM, Tridib Bandopadhyay <tridib04@gmail.com> wrote:

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

Hello all

1. When the Garbage Collector starts working can I see it.(For e.g-It
will print like Garbage Collector starting.)Somebody referred me use
GC_NOTIFY but there is no such function in the gc.c file.

The GC_NOTIFY flag is not available in ruby 1.8, it's in 1.9 only. If
you have to use ruby 1.8, you can do it yourself by adding printf
calls at the begining an end of the rb_gc_start function in gc.c.
Under 1.9 the function where GC start is called gc_collect, or at
least that's where GC_NOTIFY is placed.

2. What are the main parts in Ruby Garbage Collector I need to know to
accomplish this research.

I think this was suggested before, it will answer many of your questions.

  http://timetobleed.com/garbage-collection-slides-from-la-ruby-conference/

Regards,
Ammar

···

On Mon, Dec 6, 2010 at 4:41 PM, Tridib Bandopadhyay <tridib04@gmail.com> wrote:

On One function void rb_gc() I added one printf statement of my own.

And when I type the command -

./ruby -e 'GC.start()'

Then it is showing me my print statement.But I want to see my print
statement when i am running one of my ruby codes.

Regards

Tridib

···

--
Posted via http://www.ruby-forum.com/.

Thank You all for your responses...I am getting a clear view of the Ruby
garbage collector and the compiler...

Regards

Tridib

···

--
Posted via http://www.ruby-forum.com/.

I have some questions::--

1. Is it possible to Manually free memory(without calling GC.start) by
extending it with C? As i have tried it and its of no use for my work.

2. Can I add a new file and try to manually free memory(disabling GC)?

Regards

Tridib

···

--
Posted via http://www.ruby-forum.com/.

Thank you all for replying

What I am trying to achieve is I will allocate some 3 Bytes(for
example).. And will free that space by myself and will not bother
Garbage Collector to worry for that.I know the Ruby GC is coded in such
a way which will be helpful to the users but I want to check how much
difference it can cost if it had Manual Memory Deallocation.

Secondly, Can I call GC.start() from other file like.. within sqrt()
function in Math.c, Can I call it?If so, What is the syntax for calling
it?

···

--
Posted via http://www.ruby-forum.com/.

Jeremy Bopp wrote in post #966541:

Have you tried sprinkling some printf statements throughout gc.c
yourself yet? If the diagnostic instrumentation for gc.c is hard to
find or does not exist, adding your own can at least help you make some
progress.

To start, you could add a printf with the function name at the beginning
of each function in gc.c. Then run your build of ruby with your test
scripts to trigger garbage collection and see what gets printed. That
should give you a lead on where to add more printf calls to gc.c in
order to gather more detailed information.

-Jeremy

Yes I have tried that long back.. for example in this code fragment--

VALUE
rb_newobj()
{
    VALUE obj;
    printf("newobj"); /*My own line*/
    if (!freelist) garbage_collect();

    obj = (VALUE)freelist;
    freelist = freelist->as.free.next;
    MEMZERO((void*)obj, RVALUE, 1);
#ifdef GC_DEBUG
    RANY(obj)->file = ruby_sourcefile;
    RANY(obj)->line = ruby_sourceline;
#endif
    return obj;
}

But when I am making this change and trying to compile my ruby. It is
bombing out with a message Segmentation failure and it is printing my
print statement numerous times before bombing out.I also tried in my
other functions.

Also can you tell me when we compile the ruby with 'make' command how
does it changes the gc.c file in the executable file.

···

--
Posted via http://www.ruby-forum.com/\.

Ammar Ali wrote in post #966575:

The GC_NOTIFY flag is not available in ruby 1.8, it's in 1.9 only. If
you have to use ruby 1.8, you can do it yourself by adding printf
calls at the begining an end of the rb_gc_start function in gc.c.
Under 1.9 the function where GC start is called gc_collect, or at
least that's where GC_NOTIFY is placed.

Regards,
Ammar

I tried in there also.Here is the code.--

rb_gc_start()
{
    rb_gc();
    return Qnil;
    printf("Hello!!")
}

It is compiling but is not showing the hello message when i am running
the code.Here is my ruby code--

print "Please enter number 1 : ";
val1 = gets;
print "Please enter number 2 : ";
val2 = gets;
print "Answer : " , (val1.to_i + val2.to_i), "\n";
ct = 1;
while(ct<100)
    a=;
    na = 0;
    while(na<1000)
        a[na]=na;
        na = na+1;
    end;
    print ct;
    ct=ct+1;
end;

How to see that hello message and how to know when Garbage Collector is
doing its work.

···

--
Posted via http://www.ruby-forum.com/\.

I tried this code fragment..

i=1
while (true)
   puts "message #{i}"
   i+=1
end

But this one is ending with a message Infinity but I can't see my own
print statement when the Garbage Collector is coming...

Regards

Tridib

···

--
Posted via http://www.ruby-forum.com/.

I am sure, both can be achieved with an amount of hacking Ruby's
source code. So: yes and yes.

What I do not understand: what is the aim of your research? I mean,
people have spent numerous hours of research and engineering to make
automatic GC work, and you seem to be mainly concerned with adding
manual memory management to a language which has automatic GC. What
is the point of your research?

Kind regards

robert

···

On Tue, May 17, 2011 at 9:58 PM, Tridib Bandopadhyay <tridib04@gmail.com> wrote:

I have some questions::--

1. Is it possible to Manually free memory(without calling GC.start) by
extending it with C? As i have tried it and its of no use for my work.

2. Can I add a new file and try to manually free memory(disabling GC)?

--
remember.guy do |as, often| as.you_can - without end
http://blog.rubybestpractices.com/

It will never work. You're printing after return!

Regards,
Ammar

···

On Tue, Dec 7, 2010 at 4:27 AM, Tridib Bandopadhyay <tridib04@gmail.com> wrote:

I tried in there also.Here is the code.--

rb_gc_start()
{
rb_gc();
return Qnil;
printf("Hello!!")
}

Jeremy Bopp wrote in post #966541:

Have you tried sprinkling some printf statements throughout gc.c
yourself yet? If the diagnostic instrumentation for gc.c is hard to
find or does not exist, adding your own can at least help you make some
progress.

To start, you could add a printf with the function name at the beginning
of each function in gc.c. Then run your build of ruby with your test
scripts to trigger garbage collection and see what gets printed. That
should give you a lead on where to add more printf calls to gc.c in
order to gather more detailed information.

-Jeremy

Yes I have tried that long back.. for example in this code fragment--

VALUE
rb_newobj()
{
    VALUE obj;
    printf("newobj"); /*My own line*/
    if (!freelist) garbage_collect();

    obj = (VALUE)freelist;
    freelist = freelist->as.free.next;
    MEMZERO((void*)obj, RVALUE, 1);
#ifdef GC_DEBUG
    RANY(obj)->file = ruby_sourcefile;
    RANY(obj)->line = ruby_sourceline;
#endif
    return obj;
}

But when I am making this change and trying to compile my ruby. It is
bombing out with a message Segmentation failure and it is printing my
print statement numerous times before bombing out.I also tried in my
other functions.

Ruby creates plenty of objects during its normal operation, so this
function will be called frequently. It makes sense then that you'll see
your output many times. What doesn't make sense is the segmentation fault.

I would bet that you had more changes than just that single line in gc.c
when you built your modified version of Ruby. If that's the case, it's
likely that one of your other changes is the culprit. I suggest that
you try again from a completely fresh set of sources and add only that
single line to see if you can avoid the problem.

Also can you tell me when we compile the ruby with 'make' command how
does it changes the gc.c file in the executable file.

I'm not sure I understand what you're asking unless you're really asking
for the details of how the compiler works. Generally, the compiler is
used to compile each source file into a corresponding object (*.o) file.
Once all object files are created, they are then linked into the ruby
executable. In this case, gc.c would be compiled into gc.o which would
ultimately be linked into ruby.

The make program is usually smart enough to only rebuild the .o files
whose corresponding .c files were updated since the last build run, so
if you only change gc.c, only gc.c should be recompiled. Then the ruby
program should be recreated by linking the new gc.o file in along with
all the other .o files.

None of that really matters for you though. All you need to do is make
your changes to gc.c and then run make. You should see some output
during the make process that indicates that gc.c is being compiled and
that ultimately a new ruby file is created. Try not to get sidetracked
on the details.

-Jeremy

···

On 12/06/2010 08:19 PM, Tridib Bandopadhyay wrote:

I can see the garbage collector coming when I type.

./ruby -e 'GC.start()'

But I want to know while a code is running the printf statement written
by me should appear.

Is there any other code to make gc show my message.

This code is running for infinite time..but not showing the message.

i=1
while (true)
   puts "message #{i}"
   i+=1
end

What are the other ways?

I need to convince myself that when there is memory allocation the
Garbage collector comes for cleaning with a printf statement stating its
work.

Regards
Tridib

···

--
Posted via http://www.ruby-forum.com/.

Robert K. wrote in post #999427:

I have some questions::--

1. Is it possible to Manually free memory(without calling GC.start) by
extending it with C? As i have tried it and its of no use for my work.

2. Can I add a new file and try to manually free memory(disabling GC)?

I am sure, both can be achieved with an amount of hacking Ruby's
source code. So: yes and yes.

What do you mean by Hacking Ruby's Code... I didn't get it.. Can you
describe me in Detail.

What I do not understand: what is the aim of your research? I mean,
people have spent numerous hours of research and engineering to make
automatic GC work, and you seem to be mainly concerned with adding
manual memory management to a language which has automatic GC. What
is the point of your research?

Kind regards

robert

Yeah i know that.. But since Ruby is Using Stop the World concept. So I
was thinking of not letting the GC to come and free memory every time
for small objects,Stopping the code. So I just want to check if it can
be helpful or not...

Thanks...

Regards

Tridib

···

On Tue, May 17, 2011 at 9:58 PM, Tridib Bandopadhyay > <tridib04@gmail.com> wrote:

--
Posted via http://www.ruby-forum.com/\.

Ammar Ali wrote in post #966734:

I tried in there also.Here is the code.--

rb_gc_start()
{
rb_gc();
return Qnil;
printf("Hello!!")
}

It will never work. You're printing after return!

Regards,
Ammar

I shifted the printf statement above the return statement.Here is the
output I am getting--

./ruby doug.rb
Please enter number 1 : 3
Please enter number 2 : 4
Answer : 7
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899-bash-3.00$

Regards
Tridib

···

On Tue, Dec 7, 2010 at 4:27 AM, Tridib Bandopadhyay <tridib04@gmail.com> > wrote:

--
Posted via http://www.ruby-forum.com/\.

Maybe there are optimizations for String objects that circumvent parts
of regular garbage collection. Have you tried looking at the memory
consumption of your ruby process as you run the while loop? Is memory
usage growing over time, or does it plateau at some point?

Try using your ruby executable to run some non-trivial projects
available online. Once you find one that triggers garbage collection
the way you want it, you could try to boil that down to something more
concise.

Another suggestion would be to pick up a template library such as erb
and use that to process some large template files with random data
repeatedly. That should hopefully cause some memory growth with
something more than simple strings.

-Jeremy

···

On 12/10/2010 09:37 AM, Tridib Bandopadhyay wrote:

I can see the garbage collector coming when I type.

./ruby -e 'GC.start()'

But I want to know while a code is running the printf statement written
by me should appear.

Is there any other code to make gc show my message.

This code is running for infinite time..but not showing the message.

i=1
while (true)
   puts "message #{i}"
   i+=1
end

What are the other ways?

I need to convince myself that when there is memory allocation the
Garbage collector comes for cleaning with a printf statement stating its
work.

What do you mean by Hacking Ruby's Code... I didn't get it.. Can you
describe me in Detail.

% svn co http://svn.ruby-lang.org/repos/ruby/trunk ruby
% cd ruby
% vim ruby.c gc.c
# modify Ruby's source code here.

Yeah i know that.. But since Ruby is Using Stop the World concept. So I
was thinking of not letting the GC to come and free memory every time
for small objects,Stopping the code. So I just want to check if it can
be helpful or not...

Then start looking at other GC systems that are used by other
languages. You don't know if you can improve Ruby's GC if you don't
know about other GCs, and how they are implemented, no?

···

On Thu, May 19, 2011 at 12:43 AM, Tridib Bandopadhyay <tridib04@gmail.com> wrote:

--
Phillip Gawlowski

Though the folk I have met,
(Ah, how soon!) they forget
When I've moved on to some other place,
There may be one or two,
When I've played and passed through,
Who'll remember my song or my face.

Tridib Bandopadhyay wrote in post #999550:

So I
was thinking of not letting the GC to come and free memory every time
for small objects,Stopping the code. So I just want to check if it can
be helpful or not...

Ruby has a very primitive conservative GC -- it just scans the stack
looking for pointers. If you are a time traveler from the 1950s then
Ruby's GC might be an interesting study.

In any case it would be easy to fool Ruby's GC, if that's what you wish:
allocate a pointer and hide data behind the pointer. The GC won't see
pointers which aren't on the stack. You can manage that GC-concealed
data however you like.

···

--
Posted via http://www.ruby-forum.com/\.