Remove_const, Kernel.load, and already instantiated objects

Brent_Dillingham · 23 August 2008 01:47

Context: I'm programming a MUD in Ruby for fun and profit--if one considers that an increase in knowledge == profit.

Up to this point, I've only ever done web work with Ruby using Rails, Merb, etc., so I wanted to embark on a from-scratch project, and MUDs happen to be a favorite old pastime of mine

One of the things I'm trying to do is implement a development server that "reloads" my source files automatically when they're updated, just like Rails or Merb does. Meaning that I can connect to the MUD server and explore with my test character, while updating my source files to change behavior around on the fly.

So, reading through Merb's way of doing it, I've gathered that in order to properly "reload a class", one must use remove_const to "undefine" the Class constant, and then use Kernel.load to reload the Class' source file. Simple stuff, and with a separate thread that continuously monitors my source files for changes in their modification time, we're golden. It works great.... with one exception.

What about objects of a Class that were already instantiated -- before the Class is reloaded with its changes? Those objects are essentially still bound to the "old" Class, I've found, and do not reflect the reloaded Class's changes. This doesn't really affect Rails or Merb, since Controller and Model instances do not persist between HTTP requests, but it certainly affects a MUD, where my test character (and basically everything else) is always in the object space.

The following IRB session demonstrates: (also on pastie: http://pastie.org/258400)

     >> puts File.read('foo.rb')
     class Foo
       def hello
         "Hello, world!"
       end
     end
     => nil
     >> require 'foo'
     => true
     >> f = Foo.new
     => #<Foo:0x117fbb4>
     >> f.hello
     => "Hello, world!"

# Now I update foo.rb...

     >> puts File.read('foo.rb')
     class Foo
       def hello
         "BRAND NEW Hello World!"
       end
     end
     => nil

# Reload the file using remove_const and Kernel.load

     >> Object.send(:remove_const, :Foo)
     => Foo
     >> Kernel.load('foo.rb')
     => true

# Existing instance of Foo DOES NOT use the updated method.

>> f.hello
=> "Hello, world!"

# New object DOES use the updated method.

     >> f2 = Foo.new
     => #<Foo:0x112c6f8>
     >> f2.hello
     => "BRAND NEW Hello World!"

So here's the question -- is there some way to get an object to... erm, "reset" its Class? Sounds like evil voodoo that could potentially make things explode, but I'd truly like to see this thing work If it's just not possible though, I can live with that.

BTW -- By not doing the remove_const on the Class, I can sort of get this to work, because reloading the file with Kernel.load will effectively just reopen the class definition, adding the new changes, but also not removing methods I've deleted. And this *does* make things explode, like DataMapper for one, and I get all kinds of "already initialized constant" warnings.

The only thing I can think of is to try to finagle a way to copy existing objects into the newly defined Class, using serialization or something crazy, but that sounds really messy and probably not worth it.

Any comments? -- ideas / thoughts / similar-experience / you're-an-idiot-for-even-trying-this? Thanks!

Brent

Martin_DeMello · 23 August 2008 04:04

Not *that* messy - you basically walk the object space, marshal
everything, reload the class and unmarshal it. Works fine as long as
you don't want to restore closures.

martin

···

On Fri, Aug 22, 2008 at 6:47 PM, Brent Dillingham <brentdillingham@gmail.com> wrote:

The only thing I can think of is to try to finagle a way to copy existing
objects into the newly defined Class, using serialization or something
crazy, but that sounds really messy and probably not worth it.

_Pena_Botp1 · 23 August 2008 04:05

# ...
# >> puts File.read('foo.rb')
# class Foo
# def hello
# "Hello, world!"
# end
# end
# => nil
# >> require 'foo'
# => true

ok. you used require

# # Reload the file using remove_const and Kernel.load
# >> Object.send(:remove_const, :Foo)
# => Foo
# >> Kernel.load('foo.rb')
# => true

now you use load.

let us stick to require...

irb(main):001:0> require 'foo.rb'
=> true
irb(main):002:0> f=Foo.new
=> #<Foo:0x2900af4>
irb(main):003:0> f.hello
=> "hello"

ok

irb(main):004:0> require 'foo.rb'
=> false
irb(main):005:0> f.hello
=> "hello"

as expected, no change.
let's see if we can cheat

irb(main):006:0> $LOADED_FEATURES
=> ["e2mmap.rb", "irb/init.rb", "irb/workspace.rb", "irb/context.rb", "irb/extend-command.rb", "irb/output-method.rb", "irb/notifier.rb", "irb/slex.rb", "irb/ruby-token.rb", "irb/ruby-lex.rb", "readline.so", "irb/input-method.rb", "irb/locale.rb", "irb.rb", "irb/ext/history.rb", "irb/ext/save-history.rb", "foo.rb"]

ah

irb(main):007:0> $LOADED_FEATURES.delete "foo.rb"
=> "foo.rb"

irb(main):008:0> require 'foo.rb'
=> true
irb(main):009:0> f.hello
=> "new hello"

is that ok?
... caveat, i'm not sure if that is documented/supported. maybe, verify fr matz or nobu..

kind regards -botp

···

From: Brent Dillingham [mailto:brentdillingham@gmail.com]

Brent_Dillingham · 23 August 2008 16:05

Thanks for the replies!

Botp, your "cheat" does indeed allow you to use require again to load the source file, but this is actually doing the same thing as using Kernel.load. It's just interpreting the contents of foo.rb again, effectively reopening class Foo. Kernel.load is just like require, only require checks $LOADED_FEATURES I guess before it blindly interprets the content of your file again. Kernel.load doesn't do any such check; it interprets the file you point it to no matter what.

I did try it, but unfortunately DataMapper still complains about missing properties and such on existing objects after the reload. And the problem still remains that we are not truly "reloading" the class ... we're just reopening it, adding or overwriting methods, just like we might in an IRB session.

I did try to go the Marshal route, but Marshal.dump chokes when trying to dump most of my objects with "TypeError: can't dump hash with default proc". I don't know exactly what that's referring to (the stack trace is useless), but it probably has something to do with a DataMapper feature. Marshal seems to be pretty unreliable for any kind of non-trivial object, and I think doing any kind of "deep copy" on my objects will lead to weird duplication issues with their associations anyway. e.g. two Player objects in the same Room need to refer to the same object_id for player.room (at least, that's the assumption I'm designing under right now to prevent hitting the DB constantly). If I do a deep copy as Marshal.dump(player) would do, I'll end up with the players referring to two brand new, different copies of Room objects. Which is bad, I think.

Though it's possible that I'm making a mistake by relying on the Room objects staying in-memory. I need to read up on how garbage collection works.

At any rate, I'm about to give up on this idea unless anyone has any other suggestions. Thanks!

Brent

···

On Aug 23, 2008, at 12:05 AM, Peña, Botp wrote:

From: Brent Dillingham [mailto:brentdillingham@gmail.com]
# ...
# >> puts File.read('foo.rb')
# class Foo
# def hello
# "Hello, world!"
# end
# end
# => nil
# >> require 'foo'
# => true

ok. you used require

# # Reload the file using remove_const and Kernel.load
# >> Object.send(:remove_const, :Foo)
# => Foo
# >> Kernel.load('foo.rb')
# => true

now you use load.

let us stick to require...

irb(main):001:0> require 'foo.rb'
=> true89
irb(main):002:0> f=Foo.new
=> #<Foo:0x2900af4>
irb(main):003:0> f.hello
=> "hello"

ok

irb(main):004:0> require 'foo.rb'
=> false
irb(main):005:0> f.hello
=> "hello"

as expected, no change.
let's see if we can cheat

irb(main):006:0> $LOADED_FEATURES
=> ["e2mmap.rb", "irb/init.rb", "irb/workspace.rb", "irb/context.rb", "irb/extend-command.rb", "irb/output-method.rb", "irb/notifier.rb", "irb/slex.rb", "irb/ruby-token.rb", "irb/ruby-lex.rb", "readline.so", "irb/input-method.rb", "irb/locale.rb", "irb.rb", "irb/ext/history.rb", "irb/ext/save-history.rb", "foo.rb"]

ah

irb(main):007:0> $LOADED_FEATURES.delete "foo.rb"
=> "foo.rb"

irb(main):008:0> require 'foo.rb'
=> true
irb(main):009:0> f.hello
=> "new hello"

is that ok?
... caveat, i'm not sure if that is documented/supported. maybe, verify fr matz or nobu..

kind regards -botp

Martin_DeMello · 23 August 2008 20:33

Hm - if you're persisting to a database, what I'd recommend is
implementing a load and save feature, then doing a save/reload
class/load cycle when code changes. It will require a bit of up front
work to distinguish between game state and incidental state, but that
should improve your design too, and simplify the rest of your code
moving forward.

martin

···

On Sat, Aug 23, 2008 at 9:05 AM, Brent Dillingham <brentdillingham@gmail.com> wrote:

I did try to go the Marshal route, but Marshal.dump chokes when trying to
dump most of my objects with "TypeError: can't dump hash with default proc".
I don't know exactly what that's referring to (the stack trace is useless),
but it probably has something to do with a DataMapper feature. Marshal seems
to be pretty unreliable for any kind of non-trivial object, and I think
doing any kind of "deep copy" on my objects will lead to weird duplication

Brent_Dillingham · 24 August 2008 17:28

Yeah, I guess that's the only remaining solution.

Essentially what I'd be doing I guess is a "live reboot", where every
single game object gets recreated. And you're right that dealing with
incidental state vs. the state in the DB will be a bit hairy. But hey,
I like a good challenge And you're right, it would probably force
me to think a lot harder about the state of my game objects and what I
choose persist to the DB.

A bit saddening is that through some Googling I found that this kind
of thing seems to be a bit more feasible in Python. In python you can
actually take an existing object and do object.__class__ = MyClass
after the class is reloaded! This doesn't appear to be possible in
Ruby, but I think it's the sort of thing I was looking for initially.
It still sounds like voodoo to me, actually. I lack the low-level
knowledge to understand how that would even be implemented!

Brent

···

On Aug 23, 4:33 pm, "Martin DeMello" <martindeme...@gmail.com> wrote:

On Sat, Aug 23, 2008 at 9:05 AM, Brent Dillingham > > <brentdilling...@gmail.com> wrote:

> I did try to go the Marshal route, but Marshal.dump chokes when trying to
> dump most of my objects with "TypeError: can't dump hash with default proc".
> I don't know exactly what that's referring to (the stack trace is useless),
> but it probably has something to do with a DataMapper feature. Marshal seems
> to be pretty unreliable for any kind of non-trivial object, and I think
> doing any kind of "deep copy" on my objects will lead to weird duplication

Hm - if you're persisting to a database, what I'd recommend is
implementing a load and save feature, then doing a save/reload
class/load cycle when code changes. It will require a bit of up front
work to distinguish between game state and incidental state, but that
should improve your design too, and simplify the rest of your code
moving forward.

martin

Erik_Hollensbe · 24 August 2008 19:59

Brent Dillingham wrote:

Yeah, I guess that's the only remaining solution.

Essentially what I'd be doing I guess is a "live reboot", where every
single game object gets recreated. And you're right that dealing with
incidental state vs. the state in the DB will be a bit hairy. But hey,
I like a good challenge And you're right, it would probably force
me to think a lot harder about the state of my game objects and what I
choose persist to the DB.

Since it seems like you're trying to specialize the garbage collector,
perhaps the easiest way is to implement your own higher-level garbage
collector:

class MyObjects is a singleton which holds a persistent, mutable array
of objects with these kinds of methods:

def add(obj)
@arr.push(obj)
end

def clear
@arr.collect { |x| x.expired = true }
@arr.replace()
end

Then in your "I want these to disappear when I say so" classes:

attr_writer :expired

def initialize
@expired = false
MyObjects.add(self)
end

def some_call
raise StandardError if @expired # some form of AOP is desirable here
end

And in your "reload" code:

def reload_class
MyObjects.clear
load 'foo.rb'
end

It's ugly but it's precise, and will make the old objects complain
loudly until ruby's ready to garbage collect them, making things easier
for you to debug and manage. This will be most evident if you do any
kind of anonymous routine management and closures get involved heavily.

That said, I've never been fond of schemes that treat namespaces as
fully mutable at any time; it reeks of poor design and you end up with
schemes like this if you want to get it right.

-Erik

···

--
Posted via http://www.ruby-forum.com/\.

Erik_Hollensbe · 24 August 2008 20:04

Erik Hollensbe wrote:

It's ugly but it's precise, and will make the old objects complain
loudly until ruby's ready to garbage collect them, making things easier
for you to debug and manage. This will be most evident if you do any
kind of anonymous routine management and closures get involved heavily.

I should probably note that code in it's current state would keep
objects around that weren't intended to stay around otherwise (that is,
they would be GC'd by ruby before you did any reloading), so you'd have
to implement some form of hackneyed destructor as well.

-Erik

···

--
Posted via http://www.ruby-forum.com/\.

Topic		Replies	Views
Class reloading and constants ruby-talk	1	65	12 November 2006
Destroy object from memory.. Object.const_destroy? ruby-talk	4	93	5 July 2007
Unloading objects? ruby-talk	1	59	5 September 2006
Removing a constant definition from an environment ruby-talk	10	103	15 November 2005
Object.send(:remove_const, :Polygon) ruby-talk	2	114	31 December 2009

Remove_const, Kernel.load, and already instantiated objects

Related Topics