[ANN] Mongoose 0.1.1

ruby-talk
1

From: Jamey Cribbs [mailto:jcribbs@netpromi.com]
Sent: Thursday, July 20, 2006 1:03 PM
To: ruby-talk ML
Subject: [ANN] Mongoose 0.1.1

<snip>

*What is Mongoose?*

Mongoose is a database management system written in Ruby. It has an
ActiveRecord-like interface, uses Skiplists for its indexing, and
Marshal for its data serialization. I named it Mongoose,
because, like
Rudyard Kipling's Rikki-Tikki-Tavi, my aim is for it to be
small, quick,
and friendly.

What does Mongoose mean in terms of the future of KirbyBase? Is
KirbyBase defunct?

Regards,

Dan

This communication is the property of Qwest and may contain confidential or
privileged information. Unauthorized use of this communication is strictly
prohibited and may be unlawful. If you have received this communication
in error, please immediately notify the sender by reply e-mail and destroy
all copies of the communication and any attachments.

···

-----Original Message-----

2

Berger, Daniel wrote:

What does Mongoose mean in terms of the future of KirbyBase? Is
KirbyBase defunct?
  

I will continue to support and maintain KirbyBase. It is pretty mature and stable at this point. Some of the new things I wanted to try were going to be hard to retrofit into KirbyBase, due to it's philosophy and architecture.

My plan is to slowly bring Mongoose up to the same feature set as KirbyBase. Because of its architecture, I think Mongoose has more potential for scalability than KirbyBase.

If there is a new feature that anyone would like to see in KirbyBase, I will definitely consider it. Or, if there is a particular feature of KirbyBase that you would really like to see implemented in Mongoose, please let me know.

Jamey

3

Does †hat mean Mongoose will gain a plain text output, maybe using YAML instead of Marshal to dump?

I realize Marshal is a key element of the knew performance. I'm just wondering if there could be an option, for those who was the easily edited data and are willing to make the speed sacrifice.

James Edward Gray II

···

On Jul 20, 2006, at 2:21 PM, Jamey Cribbs wrote:

My plan is to slowly bring Mongoose up to the same feature set as KirbyBase.

4

One of the great features of KirbyBase is that the tables are
hand-editable. I recently had the need to store an Array (and could also
justify a Hash). Since these were missing from VALID_FIELD_TYPES, I
resorted to storing YAML. Perhaps a new feature could be:

- adding support for Array and Hash (maybe in the form used by irb and pp?)

Of course, each element would have to be one of the other supported types.
Maybe YAMLizing any other type would be conceivable. Just a thought...

- Dimitri

···

On 7/20/06, Jamey Cribbs <jcribbs@netpromi.com> wrote:

If there is a new feature that anyone would like to see in KirbyBase, I
will definitely consider it. Or, if there is a particular feature of
KirbyBase that you would really like to see implemented in Mongoose,
please let me know.

5

James Edward Gray II wrote:

My plan is to slowly bring Mongoose up to the same feature set as KirbyBase.

Does †hat mean Mongoose will gain a plain text output, maybe using YAML instead of Marshal to dump?

I realize Marshal is a key element of the knew performance. I'm just wondering if there could be an option, for those who was the easily edited data and are willing to make the speed sacrifice.

I think I am going to tackle this one using a two-pronged approach:

1). I will soon add #import and #export methods to the Table class, so that if you needed to edit a table as text, you could simply #export all of the records to a text file, edit it, then #import the text file back into the table. I was planning on making the format CSV (taking advantage of FasterCSV, perhaps you've heard of it :wink: ). Would people prefer YAML instead? My thought with YAML was just that, with a large table, it could quite large, since it places each field on a newline.

2). Down the road a bit, I plan on offering the option to switch off the serialization via Marshal and just have the table stored as a line-delimited plain-text file (just like KirbyBase). You would definitely lose some speed, but then you could edit the table files directly.

I would be interested in any feedback on my approach. If you would really like to see a feature be implemented sooner, please speak up!

Jamey

Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. If you are not the intended recipient(s), you are hereby notified that any dissemination, unauthorized review, use, disclosure or distribution of this email and any materials contained in any attachments is prohibited. If you receive this message in error, or are not the intended recipient(s), please immediately notify the sender by email and destroy all copies of the original message, including attachments.

···

On Jul 20, 2006, at 2:21 PM, Jamey Cribbs wrote:

6

See, I think that if you need to store an array you should transform it into
a one-to-many relation. Hashes can just be a relation (id, key, value).
Remember, a relational database is not "PStore + Searches". There's always
gonna be an issue of mapping from the objects to the db. (Hence the
existence of things like ActiveRecord and Mongoose's AR-like API.)

···

On 7/21/06, Dimitri Aivaliotis <aglarond@gmail.com> wrote:

On 7/20/06, Jamey Cribbs <jcribbs@netpromi.com> wrote:
>
> If there is a new feature that anyone would like to see in KirbyBase, I
> will definitely consider it. Or, if there is a particular feature of
> KirbyBase that you would really like to see implemented in Mongoose,
> please let me know.
>

One of the great features of KirbyBase is that the tables are
hand-editable. I recently had the need to store an Array (and could also
justify a Hash). Since these were missing from VALID_FIELD_TYPES, I
resorted to storing YAML. Perhaps a new feature could be:

- adding support for Array and Hash (maybe in the form used by irb and
pp?)

Of course, each element would have to be one of the other supported types.
Maybe YAMLizing any other type would be conceivable. Just a thought...

- Dimitri

7

Ooo, I really like this option actually. This way you keep all the speed and can still edit as needed.

My only request would be to keep the interface to this trivial to get at. It would be perfect if you could do something like:

   ruby -r mongoose -e '...DUMP CODE HERE...' > table.csv

James Edward Gray II

···

On Jul 20, 2006, at 3:10 PM, Jamey Cribbs wrote:

James Edward Gray II wrote:

On Jul 20, 2006, at 2:21 PM, Jamey Cribbs wrote:

My plan is to slowly bring Mongoose up to the same feature set as KirbyBase.

Does †hat mean Mongoose will gain a plain text output, maybe using YAML instead of Marshal to dump?

I realize Marshal is a key element of the knew performance. I'm just wondering if there could be an option, for those who was the easily edited data and are willing to make the speed sacrifice.

I think I am going to tackle this one using a two-pronged approach:

1). I will soon add #import and #export methods to the Table class, so that if you needed to edit a table as text, you could simply #export all of the records to a text file, edit it, then #import the text file back into the table. I was planning on making the format CSV (taking advantage of FasterCSV, perhaps you've heard of it :wink: ). Would people prefer YAML instead? My thought with YAML was just that, with a large table, it could quite large, since it places each field on a newline.

8

> - adding support for Array and Hash (maybe in the form used by irb and
> pp?)

See, I think that if you need to store an array you should transform it
into
a one-to-many relation. Hashes can just be a relation (id, key, value).

Ah, good point. Hadn't thought of it that way.

Remember, a relational database is not "PStore + Searches".

This I know. I just never considered KirbyBase to be a relational database.
:slight_smile:

There's always

gonna be an issue of mapping from the objects to the db. (Hence the
existence of things like ActiveRecord and Mongoose's AR-like API.)

And Og. /me goes to improve the KirbyBase support in Og...

- Dimitri

···

On 7/21/06, Logan Capaldo <logancapaldo@gmail.com> wrote:

On 7/21/06, Dimitri Aivaliotis <aglarond@gmail.com> wrote:

9

Brilliant! Great direction for you to go in my opinion. Will Mongoose be
compatible with rails at some point?

···

--
Posted via http://www.ruby-forum.com/.

10

Simon Harrison wrote:

Brilliant! Great direction for you to go in my opinion. Will Mongoose be compatible with rails at some point?
  

Boy! I could write a book about this question! I don't mean to be flippant, but it all depends on what you mean by "compatible".

***Warning*** I am not a Rails guru (I'm not a Ruby guru, for that matter). I've built a few Rails apps, but I am definitely not an expert, especially on the internals of Rails. That being said...***

If you mean, "Will there ever be an ActiveRecord adapter for Mongoose?". Hmm, that's a toughie. ActiveRecord really likes dbms's that use SQL and that implement the relational model. Assaph Mehr, the author of Pimki, actually wrote an ActiveRecord adapter for KirbyBase for Rails 1.0, called Ackbar. He had to go through quite a few code gyrations to map Rails SQL proclivities to KirbyBase's query language. It worked, but I don't know how maintainable it was. Assaph himself indicated that as ActiveRecord changed, if it broke Ackbar, it might be too much trouble to maintain. And indeed, when Rails 1.1 came out, the current version of Ackbar broke.

Now, Mongoose's api is much more ActiveRecord-like than KirbyBase's interface. Would that make it easier to write an ActiveRecord adapter for Mongoose (possible code name: Rikki :slight_smile: )? I don't know enough about writing ActiveRecord adapters to say. It might be an ironic experience, since you would possibly have to go from ActiveRecord's interface, to some quasi-SQL translation in the guts, back to an ActiveRecord-like api in Mongoose. Which leads me to my next point...

If you mean, "Will I be able to use Mongoose as a database backend for Rails with reasonable success?". Ahh, now you are on to something! Recently, I was able to get KirbyBase functioning as a Rails back end, simply by having my Rails models *not* inherit from ActiveRecord::Base. Sure, I had to write my own #find method, etc. But it worked. I was able to have it go all the way through the controller to the view and show up in the scaffold-generated code. I didn't pursue it much further, because I would have had to write a whole ActiveRecord-like frontend for KirbyBase in the model. Also, I noticed that the nice error objects that are attached to your model didn't, of course, work, since I hadn't subclassed ActiveRecord::Base.

But, this limited success leads me to believe that I might be *much* more successful using Mongoose with Rails, because Mongoose is going to stick very close to the ActiveRecord api. So, it might be as easy as having your model classes subclass Mongoose::Table, instead of ActiveRecord::Base. Things that wouldn't work would be stuff like migrations, I think. Also, it would be great if I could figure out how to get the integrated errors working like the ActiveRecord stuff does. I haven't really looked at that.

So, to, hopefully, answer your question. I think it is very possible to be able to use Mongoose as a Rails backend, with the probable loss of some functionality. Who knows, maybe in the future, ActiveRecord won't be as tied to SQL and it will be much easier to write adapters for non-SQL dbmss.

See I told you I could write a book about this!

Jamey

11

Hey Jamey-

  I love mongoose! I'm digging around in the code now to see how things work and one of the first things I miss when creating a new record is the create method that takes a hash of key => value pairs and saves a new record. So here is my first little patch for you. this goes in the Table class:

···

#-----------------------------------------------------------------------------
   # create
   #-----------------------------------------------------------------------------
   def self.create(options={})
     rec = new
     options.each do |k,v|
       rec.send("#{k}=", v) if self.column_names.include? k
     end
     rec.save
     rec
   end

It lets you do this:

irb(main):005:0> rec = Foo.create :title => 'foo title', :body => 'fooooo'
=> #<Foo:0x56d61c @body="fooooo", @id=3, @title="foo title">

  I will have a look at the query language and see if there is anything I can do to help. Is there anything specific that you would like to see from ez_where that you couldn't figure out?

Cheers-
-Ezra

12

But, this limited success leads me to believe that I might be *much*
more successful using Mongoose with Rails, because Mongoose
is going to
stick very close to the ActiveRecord api. So, it might be as easy as
having your model classes subclass Mongoose::Table, instead of
ActiveRecord::Base. Things that wouldn't work would be stuff like
migrations, I think. Also, it would be great if I could
figure out how
to get the integrated errors working like the ActiveRecord
stuff does.
I haven't really looked at that.

You've aroused my curiosity. Had a little bit of trouble figuring out
how to use mongoose (some example code would have been nice), but the
following gets some validation going. I couldn't get
validates_presence_of to work for some reason, but here's
validates_numericality_of working. Putting together a
Mongoose::ActiveTable that has most of the functionality of an
activerecord doesn't look to be that difficult. If implementations of
the rest of the methods on ActiveRecord::Base were added, you'd be able
to use validations and the other activerecord modules.

require 'mongoose'

require 'active_support'
require 'active_record/base'
require 'active_record/validations'

require 'mongoose'

require 'active_support'
require 'active_record/base'
require 'active_record/validations'

class Mongoose::ActiveTable < Mongoose::Table
  RecordNotSaved = ActiveRecord::RecordNotSaved
  def update_attribute(name, value)
    send(name.to_s + '=', value)
    save
  end
  def new_record?
    @id.nil?
  end
  def self.human_attribute_name(attribute_key_name) #:nodoc:
    attribute_key_name.humanize
  end
  def save!
    save || raise(RecordNotSaved)
  end
  
  def self.init_column(col_name, col_def, col_class)
    super
    meth = <<-END_OF_STRING
    def #{col_name}_before_type_cast
      #{col_name}
    end
    END_OF_STRING
    self.class_eval(meth)
  end
  
  include ActiveRecord::Validations
end

class Monkey < Mongoose::ActiveTable
  validates_numericality_of :arms
end

Mongoose::Table.db = Mongoose::Database.new(:path => 'c:/temp/mongoose')
unless Mongoose::Table.db.table_exists?('monkey')
  Mongoose::Table.db.create_table('monkey') do |t|
    t.add_column 'arms', :integer
  end
end

m = Monkey.new()
m.arms = '2a'
m.save!

13

There's example code distributed with Mongoose. (It's actually the same examples from KirbyBase, re written in a Mongoosian style.)

···

On Jul 20, 2006, at 11:37 PM, Daniel Sheppard wrote:

But, this limited success leads me to believe that I might be *much*
more successful using Mongoose with Rails, because Mongoose
is going to
stick very close to the ActiveRecord api. So, it might be as easy as
having your model classes subclass Mongoose::Table, instead of
ActiveRecord::Base. Things that wouldn't work would be stuff like
migrations, I think. Also, it would be great if I could
figure out how
to get the integrated errors working like the ActiveRecord
stuff does.
I haven't really looked at that.

You've aroused my curiosity. Had a little bit of trouble figuring out
how to use mongoose (some example code would have been nice), but the
following gets some validation going. I couldn't get
validates_presence_of to work for some reason, but here's
validates_numericality_of working. Putting together a
Mongoose::ActiveTable that has most of the functionality of an
activerecord doesn't look to be that difficult. If implementations of
the rest of the methods on ActiveRecord::Base were added, you'd be able
to use validations and the other activerecord modules.

14

Daniel Sheppard wrote:

You've aroused my curiosity. Had a little bit of trouble figuring out
how to use mongoose (some example code would have been nice), but the
following gets some validation going. I couldn't get
validates_presence_of to work for some reason, but here's
validates_numericality_of working. Putting together a
Mongoose::ActiveTable that has most of the functionality of an
activerecord doesn't look to be that difficult. If implementations of
the rest of the methods on ActiveRecord::Base were added, you'd be able
to use validations and the other activerecord modules.

Thanks, Daniel. This is cool. I already implemented validates_presence_of in the Table class, so that's probably why it didn't work in your class.

My goal is to implement as many methods from ActiveRecord::Base as possible. Your code gives me a great starting point for how to Mongoose interact with ActiveRecord validations.

Jamey

Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. If you are not the intended recipient(s), you are hereby notified that any dissemination, unauthorized review, use, disclosure or distribution of this email and any materials contained in any attachments is prohibited. If you receive this message in error, or are not the intended recipient(s), please immediately notify the sender by email and destroy all copies of the original message, including attachments.

15

There's example code distributed with Mongoose. (It's actually the
same examples from KirbyBase, re written in a Mongoosian style.)

Bah - I see now. That would have made it easier... I was looking for
rdoc with embedded examples and forgot to look for the 'examples'
directory. I think I figured it out from the code well enough though.