Minimizing memory allocations

module Memoize
   MEMOIZE_VERSION = "1.2.0"

Set a constant to the module version, so user code can check it as needed.

   # Memoize the method +name+. If +file+ is provided, then the method results
   # are stored on disk rather than in memory. This consumes virtually no
   # memory and is persistant.
   def memoize(name, file=nil)
      meth = method(name)

Grab a reference to the original method.

      if file
         cache = Hash.new.update(Marshal.load(File.read(file))) rescue {}
      else
         cache = {}
      end

Load an existing cache file, if requested and one exists. Otherwise, set the cache to an empty Hash.

      if file
         (class << self; self; end).class_eval do
            define_method(name) do |*args|
               unless cache.has_key?(args)
                  cache[args] = meth.call(*args)
                  File.open(file, "wb+"){|f| Marshal.dump(cache, f) }
               end
               cache[args]
            end
         end

This is the singleton class trick James Britt described. It this version, the cache is checked for an entry with all the arguments it was just called with. If it's not found, the original method is triggered to add that entry to the cache. The file is also modified to contain the new entry. Either way, the cache now has the needed entry, which is returned.

      else
         (class << self; self; end).class_eval do
            define_method(name) do |*args|
               if cache.has_key?(args)
                  cache[args]
               else
                  cache[args] ||= meth.call(*args)
               end
            end
         end
      end

Exact same thing, minus the file dump. In fact, that whole if/else/end chunk of code could be simplified to:

          (class << self; self; end).class_eval do
             define_method(name) do |*args|
                unless cache.has_key?(args)
                   cache[args] = meth.call(*args)
                   File.open(file, "wb+"){|f| Marshal.dump(cache, f) } if file
                end
                cache[args]
             end
          end

      cache

Return the cache we will use for user code to examine/modify, if desired.

   end

Hope that helps.

James Edward Gray II

···

On Jan 27, 2006, at 11:04 PM, Alex Combas wrote:

Basically, the arguments to the target method are used as a hash key into a cache. The cache hash is stored either in memory, or (optionally) on disk.

It's actually always in memory with the current implementation, and can optionally be duplicated on disk.

(But note that an in-memory hash will exist for each instance. A file allows for all instances to share values, and to have those values persist from runs of the application, but may introduce race conditions.)

The current file support doesn't really get you all the way to sharing a cache among objects, since it's only read from at the time the object is converted to use memoization.

The current file implementation is only for persistence, in my opinion, and really for just one process at that.

See the "Rethinking Memoization" for more details on all of this...

James Edward Gray II

···

On Jan 28, 2006, at 12:39 AM, James Britt wrote:

James Edward Gray II wrote:

···

On Jan 28, 2006, at 12:39 AM, James Britt wrote:

Basically, the arguments to the target method are used as a hash key into a cache. The cache hash is stored either in memory, or (optionally) on disk.

It's actually always in memory with the current implementation, and can optionally be duplicated on disk.

Ah, I missed that. Thanks.

Really nice lib, and the use of singleton methods suggests other ideas.

Thanks,

James Britt
--
http://www.ruby-doc.org - Ruby Help & Documentation
Ruby Code & Style - The Journal By & For Rubyists
http://www.jamesbritt.com - Playing with Better Toys
http://www.30secondrule.com - Building Better Tools