Mnemonic looks good, but there has been no new
version since Jan 2002. It
still uses Lapidary for its test cases. Does anyone
know of any current
projects that do similar things?
I re-implemented Prevayler for Ruby a few weeks ago,
but only the core parts (i.e, that which is in the
Java version).
Mnemonic tries to go a little further, by using proxy
methods to make object prevalence even more
transparent. Unfortunately that part of Mnemonic seems
kind of broken and messy. (I haven’t gotten any reply
from Mnemonic’s author when asking about it).
/Anders
···
=====
Anders Bengtsson ndrsbngtssn@yahoo.se
Stockholm, Sweden
Gratis e-mail resten av livet på www.yahoo.se/mail
Busenkelt!
which is a Pstore backed session manager i wrote. i’ve done quite a bit from
it and not ran into performance problems yet. it’s hard to imagine a scenario
where one would not use a rdbms or dbms (like berkley) in favour of an object
database, but require more speed than Pstore can give. it’s also really
easy to use : use it just like a regular CGI::Session only the keys and values
can be anything, not just Strings.
note that the little hack/work-around i did in the CGI class is no longer
needed in ruby 1.8.
-a
···
On Mon, 20 Jan 2003, Tim Bates wrote:
Mnemonic looks good, but there has been no new version since Jan 2002. It
still uses Lapidary for its test cases. Does anyone know of any current
projects that do similar things? RAA/Library/Database reveals nothing else
of the sort; the rest are interfaces to ‘real’ databases, rather than object
persistence. The closest I’ve seen is db-backed (which doesn’t seem to be in
RAA) which uses a ‘real’ database to implement a sort of object persistence,
but requires extending a given base class etc which Mnemonic (and Prevayler)
don’t require you to do.
–
====================================
Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================
the hard thing is then indexes… I’ve created my own little custom
wrapper on PStore that basically loads everything in memeory - but then
I have to do my own searches - I basically loop over the collection…
not good…
in time I see that I can create indexes on the things I know I need to
search on, and not keep the whole thing in memory - also speed up
searching considerably…
shouldn’t be too hard on the scale I’m talking…
of course the real problem if you have an existing database is data
migration… show me a dba who will let you move their 2 gig schema into
something the open source community has built…
my 2c
dim
Tom Sawyer wrote:
···
On Monday 20 January 2003 04:58 am, Anders Bengtsson wrote:
I’d recommend looking at Prevayler, it’s a very easy
solution to most persistence problems.
There is a Ruby port, Mnemonic, but you can write your
own Prevayler port in a few hours if you need to.
Prevayler is a very interesting idea, but it isn’t a full purpose solution. i
would love to be able to use something like this but i have tables that are
multiple hundreds of megabytes each. i’d run out of memory pretty quick.
but it did get me thinking that it would be neat is Ruby could cache its
ObjectSpace to disk, or more exactly, if it could track object usage and
cache the objects with oldest/least use. then Ruby in and of itself could be
persistent, and work as a general purpose soultion.
This idea intrigues me, but I’m grappling with what implementation would be
most useful, and whether it could be written as a library or would need to be
more built in to Ruby itself. That would depend on how it was done. Of course
the purpose is not simply to cache the objects to disk, but to do it in some
way that allows another Ruby process that could be runnning an arbitrary
amount of time later to reconstitute those objects in a useful manner. It
would also need to have flexible hooks that allow custom objects to cache
themselves as they see fit.
Hmm, so far this sounds alot like module Marshal, but with a few extra
features. What it would need that Marshal doesn’t have is the consistency
checking that Prevayler has (‘snapshots’ and transaction lists) and a fair
bit more transparency (no need to call Marshal.dump/Marshal.restore
manually). Perhaps it could be written as a wrapper around Marshal. Each
object or class that needs to be cached can ‘extend’ (to cache individual
objects) or ‘include’ (to cache every object of a particular class) a module
that flags this object as needing caching, and watches for changes in the
object’s state (somehow…) to mark as transactions. Then, on exit or
whenever else appropriate, it dumps all its info to disk to be called up
again next time around.
No implementation details yet, but does this sound useful?
Tim Bates
···
On Tue, 21 Jan 2003 12:23 pm, Tom Sawyer wrote:
but it did get me thinking that it would be neat is Ruby could cache its
ObjectSpace to disk, or more exactly, if it could track object usage and
cache the objects with oldest/least use. then Ruby in and of itself could
be persistent, and work as a general purpose soultion.
Prevayler is a very interesting idea, but it isn’t a
full purpose solution. i
would love to be able to use something like this but
i have tables that are
multiple hundreds of megabytes each. i’d run out of
memory pretty quick.
You do know that modern computers often have
hundreds of megabytes of RAM?
but it did get me thinking that it would be neat is
Ruby could cache its
ObjectSpace to disk, or more exactly, if it could
track object usage and
cache the objects with oldest/least use. then Ruby
in and of itself could be
persistent, and work as a general purpose soultion.
thought?
I’ve tried writing a few pure-Ruby solutions using
object proxies, which worked OK, but haven’t never
tried it in a production system.
It would be very interesting to add some hooks in the
ruby interpreter so that you could get some more
direct control over the objects. Python has ways of
intercepting most calls to their objects, which ZODB
(Zope’s object database) uses.
/Anders
···
=====
Anders Bengtsson ndrsbngtssn@yahoo.se
Stockholm, Sweden
Gratis e-mail resten av livet på www.yahoo.se/mail
Busenkelt!
well, i’ve been thinking about this for a bit and there seems to be a couple
of difficulties involved.
lets say to make it as transparent as possible we want to PStore the entire
ObjectSpace. one way to do this of course is to loop through the ObjectSpace
and add each object to the PStore pool, using something akin to dim’s little
Database class. the problem is then restoring these objects so everything is
just as it was. i don’t see how this restoration is possible. to simplify, it
occured to me that one might only need to persist the TopLevel which has
everything else existing under it. does that follow? you could either persist
the toplevel object itself or just the toplevel binding. we can get the
toplevel object like this:
module Kernel
def toplevel
eval(“self”, TOPLEVEL_BINDING)
end
end
but then again, how do you restore the toplevel object? is that possible?
alternately if we simply PStored the toplevel binding, we could conceivably
return to that context, but to do so we’d have to embed out entire app within
an eval statement. (note: fetch_file returns a string of a file’s contents)
ourprogram.rb
# do stuff
# all done, now store
ps = PStore.new('somefile')
ps.transaction do
ps['lue'] = TOPLEVEL_BINDING
ps.commit
end
runner.rb
# use this to run ourprogram
ps = PStore.new('somefile')
ps.transaction do
if ps['lue']
eval fetch_file('ourprogam.rb'), ps['lue']
else
eval fetch_file('ourprogam.rb')
end
end
but this is a questionable approach and rather confusing. besides, running our
entire app via an eval would probably be dog slow if it would even work at
all.
···
On Monday 20 January 2003 07:31 pm, Tim Bates wrote:
But it’s not an OODB; all you’d be doing is saving the state of your objects
at the end of the program and recalling them at the beginning - so, you could
turn it into something like an OODB by having lots of big arrays of similar
objects, but that doesn’t necessarily have to be the case. Your query
language is Ruby itself, the same way you’d query any other set of objects in
a Ruby program - you can forget that they have been cached. Your indexes can
be built yourself and cached along with the rest of your objects; attempting
to build indexes into the system will reduce its general-purpose use.
Tim Bates
···
On Tue, 21 Jan 2003 01:05 pm, Dmitri Colebatch wrote:
sounds like we’re about to write an OODB (o: as I mentioned in my other
email a query language and indexes are high on the list from my pov…
Yes, but you don’t need to persist the entire system,
just your application within the system. That way you
won’t have trouble with storing internal system stuff,
and be able to run many independent persistent
systems.
Prevayler works that way, it takes a snapshot of an
application by marshalling the top object.
sounds like we’re about to write an OODB (o: as I mentioned in my other
email a query language and indexes are high on the list from my pov…
But it’s not an OODB; all you’d be doing is saving the state of your objects
at the end of the program and recalling them at the beginning - so, you could
turn it into something like an OODB by having lots of big arrays of similar
objects, but that doesn’t necessarily have to be the case. Your query
language is Ruby itself, the same way you’d query any other set of objects in
a Ruby program - you can forget that they have been cached. Your indexes can
be built yourself and cached along with the rest of your objects; attempting
to build indexes into the system will reduce its general-purpose use.
thanks for the response. so the difference between this and PStore (or
Marshall from what I’v eheard) is …? I’m assuming you’re talking
about the load/save mechanics… at transaction boundaries writing
everything to disk? I’m already doing that in my PStore backed impl,
but am sure you’re talking about something more…
cheers
dim
···
On Tue, 21 Jan 2003 01:05 pm, Dmitri Colebatch wrote:
I don’t know, maybe I’m not; I’m just throwing ideas around. It would be more
transparent than Marshal; I haven’t looked at PStore, so maybe it already
does this, although ahoward hinted that it is intended for exclusively
web-based programming, perhaps because of the way it is tied to CGI. It’s
quite possible that something like this already exists - if someone knows of
such a beast, could you let us know since I for one would find it very
useful.
Tim Bates
···
On Tue, 21 Jan 2003 01:20 pm, Dmitri Colebatch wrote:
thanks for the response. so the difference between this and PStore (or
Marshall from what I’v eheard) is …? I’m assuming you’re talking
about the load/save mechanics… at transaction boundaries writing
everything to disk? I’m already doing that in my PStore backed impl,
but am sure you’re talking about something more…
thanks for the response. so the difference between this and PStore (or
Marshall from what I’v eheard) is …? I’m assuming you’re talking
about the load/save mechanics… at transaction boundaries writing
everything to disk? I’m already doing that in my PStore backed impl,
but am sure you’re talking about something more…
I don’t know, maybe I’m not; I’m just throwing ideas around. It would be more
transparent than Marshal; I haven’t looked at PStore, so maybe it already
does this, although ahoward hinted that it is intended for exclusively
web-based programming, perhaps because of the way it is tied to CGI. It’s
quite possible that something like this already exists - if someone knows of
such a beast, could you let us know since I for one would find it very
useful.
ok, here’s what I’ve got made more generic (developed using the well
known EDD - email driven development - technique):
class Database
def initialize(file) @store = PStore.new(file)
rollback
end
def rollback @objects = @store[‘objects’]
end
def commit @store.transaction do @store[‘objects’] = objects;
end
end
def add(object) @store.push(object)
end
… remove/save/find.
end
not sure how that fits with what you’r ethinking… but its pretty
simple and does what I need…
cheers
dim
···
On Tue, 21 Jan 2003 01:20 pm, Dmitri Colebatch wrote:
the CGI::Session::Pstore is simply plugs a Pstore into the session manager,
which ships with modules for only memory or file backed storage, both of which
are limited to string → string key-values pairs. the pstore module extends
this so cgi scripts may store object → object key-value pairs in their
session. the code is really simple but i’d be happy to answer any questions
about it.
-a
ps. apologies if you’ve already got replies regarding this, my news server is
acting really poorly today…
···
On Tue, 21 Jan 2003, Tim Bates wrote:
I don’t know, maybe I’m not; I’m just throwing ideas around. It would be
more transparent than Marshal; I haven’t looked at PStore, so maybe it
already does this, although ahoward hinted that it is intended for
exclusively web-based programming, perhaps because of the way it is tied to
CGI. It’s quite possible that something like this already exists - if
someone knows of such a beast, could you let us know since I for one would
find it very useful.
–
====================================
Ara Howard
NOAA Forecast Systems Laboratory
Information and Technology Services
Data Systems Group
R/FST 325 Broadway
Boulder, CO 80305-3328
Email: ahoward@fsl.noaa.gov
Phone: 303-497-7238
Fax: 303-497-7259
====================================
