The "perfect" ORM?

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Someone told me that this was much the way Hibernate
works in Java. I can't comment.

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

In case anyone is morbidly curious, I want this so that
I can continue work on Tycho without getting bogged
down in data storage issues. (Someone teased me at the
conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

And who if anyone is interested in working on this
project?

Thanks,
Hal

Hal Fulton wrote:
...

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

What do you mean by 'metadata'? The data types of your object's state?

James

···

--

http://www.ruby-doc.org - The Ruby Documentation Site
http://www.rubyxml.com - News, Articles, and Listings for Ruby & XML
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys

Ideally you'd be more generic than an ORM.

You'd start with objects.

1.Have some method of querying them ( and implicit in this is some
concept of unique ID/PK ).
2.Have some method of storing/retrieving them.
3.Have some form of collection which can contain the full object set
or a subset.

You could go overboard by being very abstract for 1&2 which would
allow you to utilise the storage engines query facility ( map to SQL
for example), or very low-level rubyesque ( blocks for filters on
properties etc ).

I don't think you should assume a RDBMS as the persistence mechanism.

Also, db4o is worth looking at, it uses QBE.

cheers
lyndon

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Someone told me that this was much the way Hibernate
works in Java. I can't comment.

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

In case anyone is morbidly curious, I want this so that
I can continue work on Tycho without getting bogged
down in data storage issues. (Someone teased me at the
conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

And who if anyone is interested in working on this
project?

I greatly specify code defining the database (Og)
versus having to deal with the DB directly (AR).

Using ruby's capabilities it would be fairly easy to create
a completely implicit ORM (obj.instance_variables etc.) even
without subclassing --this is not really a problem. The
problem is the corner cases where, for example, an instance
variable contains transient data which is not useful (or even
counterproductive) when stored so typically some way of either
opting in or out is desirable to have. The other issue may be
managing relationships between the stored objects.

That said, I would be happy to contribute what I can
towards such an interface.

Thanks,
Hal

E

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

I've spent a fair bit of time with Hibernate and I can safely say that it is not the "ruby way" (even from the little experience I have with ruby)

Here's some basic example code for you to look at anyhoo.

Java class:

class Cat

  private long id;
  private String name;

  public long getId() {
    return this.id;
  }

  /** note that this is the id field that Hibernate uses, it should not be directly setable by external clients - hence private */
  private void setId(long id) {
    this.id = id;
  }

  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }
}

Now you have a simple JavaBean style of class, the only thing that Hibernate imposes here is that it seems to be easier to use id values that are auto-generated (long maps to number(19) in Oracle etc). You can use Strings (auto generated hex values) or assign the id/primary key yourself. Best practice in the Hibernate community is to let the database auto-generate where possible and to always use surrogate keys.

So yes in the raw Java code for the model, Hibernate does not interfere. However at this point, you still need to map the JavaBean to the database table. This is done with a (verbose) xml mapping file. As these are such a pain to write, most people use XDoclet to generate the mapping automatically. For XDoclet to do this, you have to sprinkle attributes into your Java code like fairy dust. So the code would really look like...

/**
* @hibernate.class
* table="cat"
*/
class Cat

  private long id;
  private String name;

  /**
   * @hibernate.id
   * column="cat_id"
   * generator-class="sequence"
   */
  public long getId() {
    return this.id;
  }

  /** note that this is the id field that Hibernate uses, it should not be directly setable by external clients - hence private */
  private void setId(long id) {
    this.id = id;
  }

  /**
   * @hibernate.property
   * column="name"
   */
  public String getName() {
    return name;
  }

  public void setName(String name) {
    this.name = name;
  }
}

In Java5, the introduction of annotations allows these special @whatevers to be placed outside of comments. Hibernate3 supports both styles (in comment and true annotations). If you can say that these Attribute/Annotations don't couple themselves to your model code, then yes, the assertion that Hibernate is unobtrusive is true. On the other hand, actually keeping the metadata in a separate file (xml mapping in Hibernates case), means that the turn-around on a change is fairly significant. Trust me, coding up a Hibernate app without using Ant + XDoclet is an exercise in pain, even with Ant + XDoclet, the change code->deploy is still a drag.

There are some cool parts of Hibernate, being completely flexible on how you configure every aspect is probably the most 'enterprise' feature of it. It allows it to be used so much more easily with legacy data (composite business keys, wierd table structures etc).

Right now, I'd say that Hibernate3 + Spring + J2EE services are very useful when you have to actually build an enterprise application (access n datastores of various forms across different locations etc). But ActiveRecord (and the rest of Rails+Ruby) is a much more efficient (in terms of coding time) way of getting to 80% of the functionality of Hibernate3/EJB3 that it doesn't make sense to use Hibernate in all cases.

Erm wandered of the point there a little.

Summary
Hibernate is very good at allowing you to specify everything, but you pay the price with overly complex and verbose configuration files that *must* be in sync with your model code for the application to work - this synchronization issue is the achilles heal of Hibernate in my experience - I've wasted too much time when the server has cached an old mapping file instead of deploying the new one.

Kev

I think Og has a very good _outward_ design. I'm trying to get George
to trim it up and get rid of the intrusiveness. If this were done I
think Og would be pretty close to perfect.

T.

Hahahaha ... oh, that's a good one :slight_smile:

Actually, hibernate is very flexible, but you do end up specifying a lot of
metadata either through annotations, comments (as another poster
demonstrated), or via XML files. Much more intrusive than, say,
ActiveRecord.

···

On Tuesday 25 October 2005 08:11 pm, Hal Fulton wrote:

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Someone told me that this was much the way Hibernate
works in Java.

--
-- Jim Weirich jim@weirichhouse.org http://onestepback.org
-----------------------------------------------------------------
"Beware of bugs in the above code; I have only proved it correct,
not tried it." -- Donald Knuth (in a memo to Peter van Emde Boas)

I've been looking in to the various options available to us Ruby developers as well. A coworker (Troy Heninger) and I are looking at implementing a "knowledge base" or ODBMS, but I haven't worked out the details yet. Troy is much more knowledgeable in this area.

If you're interested, the 3 interesting packages I've found so far are:

Purple - http://purple.rubyforge.org/
DyBase - DyBASE - Object Oriented Database for Languages with Dynamic Type Checking
Madeleine - http://madeleine.sourceforge.net/

Duane Johnson
(canadaduane)

···

On Oct 25, 2005, at 5:11 PM, Hal Fulton wrote:

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Someone told me that this was much the way Hibernate
works in Java. I can't comment.

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

In case anyone is morbidly curious, I want this so that
I can continue work on Tycho without getting bogged
down in data storage issues. (Someone teased me at the
conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

And who if anyone is interested in working on this
project?

Thanks,
Hal

orm = KSDatabase.new('kirbybase:///var/db/msds',:pollute => true)

sodiums = Chemicals.chemical_name.like('sodium')
chlorides = Chemicals.chemical_name.like('chlorides')
sodium_chlorides = sodiums & chlorides

or

sodium_chlorides = Chemicals.select do |c|
  (c.chemical_name =~ 'sodium') & (c.chemical_name =~ 'chloride)
end

Chemicals.new({:chemical_name => 'potassium chloride'})

schools_with_nacl = orm.select(:Schools, :Inventories, :Chemicals) do |s,i,c|
  (c.chemical_name == 'sodium chloride') &
  (i.chemical_idx == c.idx) &
  (i.school_idx == s.idx)
end

If there is sufficient meta data in Kirbybase to identify relationships between tables (such as can be done in some dbs with foregin key constraints and the like):

schools_with_nacl = Chemicals.chemical_name.is('sodium chloride').inventories.collect {|i| i.school}.uniq

Otherwise one would have to specifically declare relationships:

Chemicals.to_many(:relationship => :inventories, :foreign_table => :inventories, :foreign_key => :chemical_idx)
Inventories.to_one(:relationship => :school, :foreign_table => :school, :local_key => :school_idx)

Which, using just a smidge of convention over configuration logic, could be written as simply as:

Chemicals.to_many(:relationship => :inventories)
Inventories.to_one(:relationsip => :school)

Now, to be honest, you can't do this, quite like this, yet. As Kansas works right now, database connection is uglier, there is no adaptor for KirbyBase, relationships must be manually declared either with that mechanism or with a class declaration, and a few other things. However, a couple days ago I started gutting parts of Kansas to modularize them, clean up interfaces, sniff metadata and act on it better, and make it easy to do things like write adaptors to non-SQL data sources like KirbyBase, or to give better optimization of generated queries on databases that allow it, such as by using hinting with Oracle queries, for example.

The above examples come directly from my current plan of how I want the library to work, based on my needs and the input that I have gotten from others. It's completely subject to change from internal or external influence at this point, as I'm still working on the modularization of the query generation/db interface code. The motivation for this, quite honestly, is so that I can have an adaptor to KirbyBase or even directly to a directory of CSV files which can be treated as a database of tables, or to other non-db data sources.

Kirk Haines

···

On Tuesday 25 October 2005 6:11 pm, Hal Fulton wrote:

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

#: Hal Fulton changed the world a bit at a time by saying on 10/26/2005 2:11 AM :#

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Someone told me that this was much the way Hibernate
works in Java. I can't comment.

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

In case anyone is morbidly curious, I want this so that
I can continue work on Tycho without getting bogged
down in data storage issues. (Someone teased me at the
conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

And who if anyone is interested in working on this
project?

Thanks,
Hal

Hi Hal!

I've been working with Hibernate for quite a while and imo it is correctly approaching so called object - relational mismatch.

The real good thing about this approach is that it is not obtrusive in any ways with your domain model objects and it let's you focus and work only on the objectual world.

On the dark side of the problem: you should provide in some way the mapping between the object world and the relational world. While there are a few things that could be a little simplified (like automatic type conversions), the big problem is the impossibility to use this simplified form on relationships. If the parametrized types would have been implemented without the erasure mechanism than this simplification could be brought further, but for the moment we have to use some other way to describe relations: and here comes into play the metadata. There are a few different approaches used: metadata through external XML, metadata through javadoc comments and lately metadata through annotations.
There have been long discussion about the benefits and pitfalls of each of these approaches, so I will not enter this discussion here.

hth,

./alex

···

--
.w( the_mindstorm )p.

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Do you want an ORM? or do you want a way to persist classes in a non-intrusive way? Be careful what you ask for :slight_smile:

I am putting the final touches on a project (<http://rubyforge.org/projects/xampl/&gt;\) that I've been working on for a while now. It has its roots in a Java tool that I've been working on since 1998 or so and that has been used in eight or nine quite large commercial products (500k to 2000k lines of code). There is a Common Lisp version as well. I am working through a small but relatively complex example (in Ruby) just to make sure I've not missed anything that Ruby needs (and a good thing I did too).

This will address some of the issues you raise and that you've mentioned in previous posts.

It is unobtrusive as long as you play along. Persistence is only one of the goals of the tool, it is also trying to provide a useful framework for projects that use it.

There is more information on my weblog in the ruby category <http://recursive.ca/hutch/index.php?cat=16&gt;, a few additional articles in the xampl category talking about the Java or CL version, if you are curious. The articles mostly talk about xampl as an XML binding tool -- which it also does do.

Right now, xampl is targeted at new code. Fitting it into existing code can be done but requires some familiarity with the tool, and there is no guarantee that it would be all that useful in the end.

···

On Oct 25, 2005, at 8:11 PM, Hal Fulton wrote:

Someone told me that this was much the way Hibernate
works in Java. I can't comment.

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

In case anyone is morbidly curious, I want this so that
I can continue work on Tycho without getting bogged
down in data storage issues. (Someone teased me at the
conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

And who if anyone is interested in working on this
project?

Thanks,
Hal

----
Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/&gt;
Recursive Design Inc. -- <http://www.recursive.ca/&gt;
Raconteur -- <http://www.raconteur.info/&gt;

I'm not sure if this is relevant but in my opinion the perfect ORM is no ORM.

Why do we think we need ORM? Why do we build wonderful things like Rails?

Because our applications have objects and we need to persist those objects in a useful way. We want to be able to find those objects and change them and save them.

Personally, I'm willing to sacrifice a LOT to get really simple object persistence. Then again I'm not writing applications that need to handle tens of thousands of object finds every second.

I would love to be able to do something like:

class Person < ActiveObject::Base
     field :last_name, String
     field :first_name, String
     has_many :aliases, String
     transient :some_transient_value
     belongs_to :team
     has_many_ordered :roles

     #methods and the like
end

This would be enough to create a basic object, its schema, and everything else. We might also do:

   field :last_name, String, validate_max_length( 55 ), validate_min_length( 10 )

After creating your class you would have to run some sort of generation script to create your database tables. Beyond that, you would get an object that works (in my perfect world) exactly like ActiveRecord objects do.

I realize that this makes certain common database optimizations impossible. I'm not sure that's a problem, like I said, I don't do large scale applications.

Adam

···

On 25-Oct-05, at 5:11 PM, Hal Fulton wrote:

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).

Someone told me that this was much the way Hibernate
works in Java. I can't comment.

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

Logan's KirbyRecord was a step in the right direction,
it seemed to me. But it was like ActiveRecord -- it
forced you to store your metadata in your own objects.

Og is cool, but is even more intrusive.

In case anyone is morbidly curious, I want this so that
I can continue work on Tycho without getting bogged
down in data storage issues. (Someone teased me at the
conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Anyhow: Who knows Hibernate and can comment on its
usefulness as a model?

And who if anyone is interested in working on this
project?

Thanks,
Hal

So anyway, this is one of my highest priorities -- to
make an ORM (that works the way I like) to wrap
KirbyBase. (With additional code, it should/could wrap
any other db, of course.)

FYI, the development release of Og includes a KirbyBase wrapper.

Og is cool, but is even more intrusive.

Why is Og intrusive? can you elaborate?

···

--
http://www.gmosx.com
http://www.navel.gr

(Someone teased me at the

conference that a year had gone by, and Tycho still
hadn't progressed any. And he wonders how that spider
got in his salad.)

Hehe. That was me. And the spider...hey! That was you!?

Dan Amelang

I don't know if you've seen this one or not.
But I think a natural OO/Ruby interface for persistance on top of SQLite
for the backend could be interesting.

Here is one that is being done in Python.
Looks interesting.

http://divmod.org/trac/wiki/DivmodAxiom

Jimmie

Jim Weirich wrote:

···

On Tuesday 25 October 2005 08:11 pm, Hal Fulton wrote:

For many weeks I have had this at the back of my mind.

I want a really good ORM that is highly non-intrusive
(e.g., I don't have to inherit and I don't have to
clutter my classes and objects with metadata).
   

Actually re-thinking this again,

If you don't want to inherit from a base class (a la ActiveRecord), could you build some kind of Dependency Injection to use ActiveRecord without having to inherit?

Kev

This guys stuff is very good. I've not used dybase, but I have used several of his other tools (have a look around his site, it is amazing what this one guy has done).

···

On Oct 25, 2005, at 11:18 PM, Duane Johnson wrote:

DyBase - DyBASE - Object Oriented Database for Languages with Dynamic Type Checking

----
Bob Hutchison -- blogs at <http://www.recursive.ca/hutch/&gt;
Recursive Design Inc. -- <http://www.recursive.ca/&gt;
Raconteur -- <http://www.raconteur.info/&gt;

James Britt wrote:

What do you mean by 'metadata'? The data types of your object's state?

Basically, yes.

Hal

Lyndon Samson wrote:

Ideally you'd be more generic than an ORM.

True, but I'd want to walk before I run, so to speak.

1.Have some method of querying them ( and implicit in this is some
concept of unique ID/PK ).
2.Have some method of storing/retrieving them.
3.Have some form of collection which can contain the full object set
or a subset.

That's a good analysis.

You could go overboard by being very abstract for 1&2 which would
allow you to utilise the storage engines query facility ( map to SQL
for example), or very low-level rubyesque ( blocks for filters on
properties etc ).

Mmm, again I wouldn't get too fancy in the early iterations.

I don't think you should assume a RDBMS as the persistence mechanism.

Maybe not, but I'd want to start with some kind of backend that already

worked, as opposed to creating both a backend and a frontend.

Also, db4o is worth looking at, it uses QBE.

I've never heard of it, but I will Google.

Thanks,
Hal

ES wrote:

Using ruby's capabilities it would be fairly easy to create
a completely implicit ORM (obj.instance_variables etc.) even
without subclassing --this is not really a problem. The
problem is the corner cases where, for example, an instance
variable contains transient data which is not useful (or even
counterproductive) when stored so typically some way of either
opting in or out is desirable to have. The other issue may be
managing relationships between the stored objects.

Yes. The latter problem is worse than the former, I think. And
there are likely others I haven't seen yet.

KirbyBase has the concept of "calculated fields" which might
help relieve us of storing certain fields.

Thanks,
Hal