[Project Idea] RCM - A Ruby Configuration Management System

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

Regards,

  Michael

Michael Neumann ha scritto:

Hi all,

Is there anyone interested in joining a project that aims at creating a Monotone (www.venge.net/monotone) inspired distributed configuration management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code quality, from which further ideas could be implemented more easily than is the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

have you considered joining forces with Zed Shaw's FastCST?
I don't know what features you're planning to take out from monotone, but since FCST is based on the idea of plugins to add functionalities, I imagine you could possibly add them..

Michael Neumann said:

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

It sounds interesting. I'd need to read more about monotone and do some
comparisions with other system, but this does make me wonder: do we really
need ANOTHER version control system? Based on the page on the Monotone
site (http://www.venge.net/monotone/others.html\) there are already a ton
(and that list is just the free ones.)

But at the same time there are tons of web frameworks and Rails is
certainly an improvement on many of them...

So if you move forward with this, contact me as I could probably help out.

Ryan

Michael Neumann wrote:

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

I don’t want to poop on your parade, but isn’t the
SCM/RCM/whatevergoddamacbbreviationwereusingtodayforit market kind of
flooded at the moment? What do you hope to bring to it that isn’t
already being brought to you in GNU Arch (1|2|Bazaar(-ng)?), Darcs, git,
and (to a lesser extent) Monotone, subversion, Aegis, svk, cvs, OpenCM,
Perforce, Visual Sourcesafe.

This is another brand of software that everyone sooner or later writes
their own version of, along with text editors, IRC/FTP clients, and
programming languages, yet no one seems to get right,
        nikolai

···

--
Nikolai Weibull: now available free of charge at http://bitwi.se/\!
Born in Chicago, IL USA; currently residing in Gothenburg, Sweden.
main(){printf(&linux["\021%six\012\0"],(linux)["have"]+"fun"-97);}

Hi Everyone,

Already replied to you (Michael) offline, but thought I'd throw this out
there.

I know I've been totally delinquent in the FastCST development, but I
promise I am actually working on it, just kind of indirectly.

* I'm currently polishing off the Ruby/Odeum code so that I can use it
in FastCST for full text search over code and revisions. This should
replace just about all the tagging and "revisions follow moves" stuff
out there. My recent tests show that this approach just rocks.
* I'm finishing off the CookbooXUL application since I want to provide a
nice XUL based interface to FastCST.
* I'm working on the Linda distributed stuff so that I can include
zero-config setup.
* And I'm doing a libevent binding so...I can do a libevent binding.

Anyway, just thought I'd throw that out so folks don't lose hope.
Anyone who's interested in kicking me in the butt and motivating me to
actually work with them on FastCST should send a swift kick to my e-mail
address.

Zed

···

On Tue, 2005-06-28 at 01:11 +0900, Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

Regards,

  Michael

Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

This wouldn't make any sense for the development of the interpreter,
but for library level code, it would be interesting to consider a
Ruby-specific approach: rather than model a file in terms of lines of
text, model it as Ruby class and method definitions, and do all of the
merging and diffing based on that. What you lose is any ability to
version other kinds of code, like associated C or YAML files. But you
gain a much simpler and cleaner model (almost all of the hard, annoying
problems when writing an SCM have to do with textual diffs; take that
away and you're left with the fun conceptual part), and a much
friendlier tool to use: spurious merge conflicts become a thing of the
past; you can autogenerate meaningful, detailed changelogs; you can
properly track changes that a traditional SCM would never notice
(forget file renaming, think about moving a single method or class
between two files...).

This is the approach we took when building the Monticello version
control system for Squeak. The first version, which is currently in
widespread use (hundreds of packages and several large commercial
projects), has a very similar design to monotone (unfortunately we
hadn't seen monotone yet when we wrote it, and so didn't think to use
public key crypto the way monotone does, which is too bad because it's
a very cool idea). The second version, which is only now nearing
usability, is much closer to Bram Cohen's Codeville - which has turned
out to be a great model in terms of code elegance and potential
flexibility, although we've yet to see how it works out in real-world
use.

Of course it's easier to take this approach for Smalltalk (which
already has a method-level IDE) than it would be for Ruby, but since
the SCM only needs to "understand" Ruby code at about the same level as
RDoc, it shouldn't be all that bad either.

I don't really expect people to jump on this bandwagon - I realize that
part of the appeal of Ruby over Smalltalk is that it lives firmly in
the world of text files, and this would take it slightly away from
that. But I thought I'd give it a shot anyway. :slight_smile:

Avi

Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

I'd rather Python won this race, if it meant we could get you to deal
with the 11 bugs, 4 feature requests and 3 patches on the Ruby DBI
project.

Regards,

Dan

Michael Neumann ha scritto:
> Hi all,
>
> Is there anyone interested in joining a project that aims at creating a
> Monotone (www.venge.net/monotone) inspired distributed configuration
> management system in Ruby?
>
> Major goal is clean and understandable code, good abstractions and high
> code quality, from which further ideas could be implemented more easily
> than is the case for Monotone (at least for me).
>
> Ultimate goal of course is the switch from CVS to RCM for the development
> of the ruby interpreter :wink:
>
> Or do we want to let Python win the race? :wink:

have you considered joining forces with Zed Shaw's FastCST?

Yes, I wrote him an email just before writing this.

I don't know what features you're planning to take out from monotone,
but since FCST is based on the idea of plugins to add functionalities, I
imagine you could possibly add them..

Mainly how Monotone works internally. But the plugin idea is definitly an
interesting idea...

Main problem is that it has to be accepted and used by a couple of people.
Using a CM system alone is no fun.

Regards,

  Michael

···

Am Monday 27 June 2005 18:35 schrieb gabriele renzi:

Michael Neumann said:
> Is there anyone interested in joining a project that aims at creating a
> Monotone (www.venge.net/monotone) inspired distributed configuration
> management system in Ruby?

It sounds interesting. I'd need to read more about monotone and do some
comparisions with other system, but this does make me wonder: do we really
need ANOTHER version control system? Based on the page on the Monotone
site (http://www.venge.net/monotone/others.html\) there are already a ton
(and that list is just the free ones.)

You're right. There are for sure many good enough version control systems. The
best I ever used was OpenCM (www.opencm.org), but development has
discontinued. Monotone comes close to OpenCM. But then I heard (and
confirmed) that checking out the Monotone repository takes ~90 minutes. This
and the simplicity of the Monotone model inspired me in writing my own :wink:

Some personal comments about the other versioning systems:

Subversion: the hell from an administrative point of view. you need shell
accounts for each commiter, or you accept that the data goes unencrypted over
the wire, or you setup an Apache with all the required modules, and chown the
whole repository to the www user.

Darcs: Very nice, but using it as a central repository requires again local
machine permissions.

OpenCM: Uses cryptography to ensure data integrity. Easy to administrate, as
users and groups are managed by the repository (no unix access rights
required). Advanced features were planned (but never integrated into the
downloadable version): Disconnected Commits, Replication.

Monotone: Uses cryptography for data integrity. Unlike OpenCM, this is a
distributed system.

But at the same time there are tons of web frameworks and Rails is
certainly an improvement on many of them...

So if you move forward with this, contact me as I could probably help out.

Great! There are lots of small tasks to do, but very little time to do it all
myself.

Regards,

  Michael

···

Am Monday 27 June 2005 19:42 schrieb Ryan Leavengood:

Before I finished that sentence, I expected you to say, "Anyone who's
interested in kicking me in the butt and motivating me to actually
work with them on FastCST should send a [fifty dollar check] to my ...
address." :slight_smile:

I thought, "Yeah, that'd do it."

···

On 6/27/05, Zed A. Shaw <zedshaw@zedshaw.com> wrote:

Anyone who's interested in kicking me in the butt and motivating me to
actually work with them on FastCST should send a swift kick to my e-mail
address.

--
Regards,
John Wilger

-----------
Alice came to a fork in the road. "Which road do I take?" she asked.
"Where do you want to go?" responded the Cheshire cat.
"I don't know," Alice answered.
"Then," said the cat, "it doesn't matter."
- Lewis Carrol, Alice in Wonderland

Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Hi Avi,

This wouldn't make any sense for the development of the interpreter,
but for library level code, it would be interesting to consider a
Ruby-specific approach: rather than model a file in terms of lines of
text, model it as Ruby class and method definitions, and do all of the
merging and diffing based on that. What you lose is any ability to
version other kinds of code, like associated C or YAML files. But you
gain a much simpler and cleaner model (almost all of the hard, annoying
problems when writing an SCM have to do with textual diffs; take that
away and you're left with the fun conceptual part), and a much
friendlier tool to use: spurious merge conflicts become a thing of the
past; you can autogenerate meaningful, detailed changelogs; you can
properly track changes that a traditional SCM would never notice
(forget file renaming, think about moving a single method or class
between two files...).

The core of RCM would allow this kind of SCM as it knows nothing about files, only entities and their names. In the case of files, the name would be the file name, and the entity the content of the file. But in the same way, you could version classes and methods.

BTW, a had a similar idea some time ago, but then realized, that in Java or Smalltalk (where you already store classes/methods in the image), this is easy, but how would you version this Ruby source code:

   class A
     attr_accessor :a, :b

     eval %{
       ...
     }
   end

Ruby is simply too dynamic :wink:
And I'm not sure whether I'd like to store:

   def a
     @a
   end

   def a=(v)
     @a = v
   end

instead of attr_accessor :a, :b

Smalltalk evaluates those things only once, and then the original code that generated those methods is gone.

Or how would you split this file as methods, classes etc.?

   class A
     attr_accessor :a

     def b
     end

     C = 4
   end

You can't simply reorder the statements inside the class!

This is the approach we took when building the Monticello version
control system for Squeak. The first version, which is currently in
widespread use (hundreds of packages and several large commercial
projects), has a very similar design to monotone (unfortunately we
hadn't seen monotone yet when we wrote it, and so didn't think to use
public key crypto the way monotone does, which is too bad because it's
a very cool idea). The second version, which is only now nearing
usability, is much closer to Bram Cohen's Codeville - which has turned
out to be a great model in terms of code elegance and potential
flexibility, although we've yet to see how it works out in real-world
use.

Do you know if Codeville is usable for C code? Or is a very regular object structure (packages, classes, methods) required?

Of course it's easier to take this approach for Smalltalk (which
already has a method-level IDE) than it would be for Ruby, but since
the SCM only needs to "understand" Ruby code at about the same level as
RDoc, it shouldn't be all that bad either.

I fear, we'd loose some kind of dynamism. Another approach is of course, to store diffs with knowledge of Ruby's syntax. Sure, you're approach would be much more elegant.

I don't really expect people to jump on this bandwagon - I realize that
part of the appeal of Ruby over Smalltalk is that it lives firmly in
the world of text files, and this would take it slightly away from
that. But I thought I'd give it a shot anyway. :slight_smile:

Thanks for sharing those ideas with us.

Regards,

   Michael

···

avi.bryant@gmail.com wrote:

Zed A. Shaw wrote:

Hi Everyone,

Already replied to you (Michael) offline, but thought I'd throw this out
there.

I know I've been totally delinquent in the FastCST development, but I
promise I am actually working on it, just kind of indirectly.

* I'm currently polishing off the Ruby/Odeum code so that I can use it
in FastCST for full text search over code and revisions. This should
replace just about all the tagging and "revisions follow moves" stuff
out there. My recent tests show that this approach just rocks.
* I'm finishing off the CookbooXUL application since I want to provide a
nice XUL based interface to FastCST.
* I'm working on the Linda distributed stuff so that I can include
zero-config setup.
* And I'm doing a libevent binding so...I can do a libevent binding.

Wow! All that sounds impressive. Please, keep us informed.

Could you explain more detailed what you mean with: ...tagging and "revisions follow moves"?

Could you give a practical example (e.g. how it would work on the command line), so that I can understand it? :wink:

Regards,

   Michael

Daniel Berger wrote:

Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

I'd rather Python won this race, if it meant we could get you to deal
with the 11 bugs, 4 feature requests and 3 patches on the Ruby DBI
project.

Sorry, but I'm no more actively maintaining Ruby/DBI. I should have better announced this before. Anyone, who is still using Ruby/DBI, would like to overtake the project?

Regards,

   Michael

The core of RCM would allow this kind of SCM as it knows nothing about
files, only entities and their names. In the case of files, the name
would be the file name, and the entity the content of the file. But in
the same way, you could version classes and methods.

Good. But the important thing here is that if your entities are small
enough, you don't need to worry about automatic merging or diffing
within the content of the entity. You only need to compare different
content versions for equality. This makes a *huge* difference to the
complexity of the SCM. So it's not really about whether you could
implement something method-based on top of something that was file
based (clearly you could), but how much work and complexity you can
avoid by doing it method-based from the start.

Or how would you split this file as methods, classes etc.?

   class A
     attr_accessor :a

     def b
     end

     C = 4
   end

You can't simply reorder the statements inside the class!

That's definitely the most challenging problem of this approach. To
start with you could have an entity (maybe represent it as a singleton
method called "initialize" on the class, if you like) that would
gather up all of the "loose" code inside the class def. In most
cases, I think just keeping this in a single block at the top of the
class def would be ok. However, you would definitely need to provide
some structure for cases where ordering was important. One TSTTCPW
strategy would be to use RDoc-like comments to mark off blocks of code
that should be treated as individual entities, combined with a general
inclination on the part of the SCM to keep entities in the same order
where possible. So:

class A
#RCM
attr_accessor :a
#/RCM

def b
end

#RCM
C = 4
#/RCM
end

Do you know if Codeville is usable for C code? Or is a very regular
object structure (packages, classes, methods) required?

Codeville doesn't require any structure, it's line-based. Monticello
2 uses the Codeville approach (or at least I think it's the same, we
came to it independently) of a 2-way merge + independent ancestry info
for each fine-grained entity, but in Codeville's case the entity is
the line whereas in Monticello's case it's the method. I do think the
idea works much *better* if there's a regular structure (how do you
uniquely identify a line to attach ancestry to it?) but I think it's
still worth looking at even for a more language-agnostic system.

One thing that's really cool about this, though I don't know if
Codeville itself takes advantage of it, is that you can do a perfect
merge having *only* the two versions you're merging plus their
metadata. So a formal repository becomes completely unnecessary: you
never need to look up any previous history. You could build this as
an extension to RubyGems, for example: each gem version would carry
around enough info to let it be merged into any other gem version, and
they could just be emailed or FTP'd or whatever around without any
special protocol. It's about as distributed as it gets.

Monticello 1 did this as well, but with a Monotone-style three-way
merge, which means that you still need enough of a repository to find
and retrieve the common ancestor.

Cheers,
Avi

···

On 6/28/05, Michael Neumann <mneumann@ntecs.de> wrote:

Michael Neumann wrote:

Zed A. Shaw wrote:

Hi Everyone,

Already replied to you (Michael) offline, but thought I'd throw this out
there.

I know I've been totally delinquent in the FastCST development, but I
promise I am actually working on it, just kind of indirectly.

* I'm currently polishing off the Ruby/Odeum code so that I can use it
in FastCST for full text search over code and revisions. This should
replace just about all the tagging and "revisions follow moves" stuff
out there. My recent tests show that this approach just rocks.
* I'm finishing off the CookbooXUL application since I want to provide a
nice XUL based interface to FastCST.
* I'm working on the Linda distributed stuff so that I can include
zero-config setup.
* And I'm doing a libevent binding so...I can do a libevent binding.

Wow! All that sounds impressive. Please, keep us informed.

Could you explain more detailed what you mean with: ...tagging and "revisions follow moves"?

Could you give a practical example (e.g. how it would work on the command line), so that I can understand it? :wink:

Also, the use of Linda for zero config setup sounds interesting. Care to elaborate a bit, or should we wait for the release?

Yeah, basically a common problem is that someone moves/renames a file
and then changes it, and then behaves as a very poor development citizen
and checks the changes in without warning people. Everyone else then
starts using code that has moved out from under them and needs to find
out what happened to the missing code.

In CVS you're just out of luck. File go *poof*.

Other systems have now started giving you their own mv commands so that
they can track your move operations and then keep the revision history
moving along.

What I've done is approached the problem as a generalization of
searching your source code. I consider the above "dammit, where did
that file go." problem and other "revision follows move/rename" problems
as a search problem.

If I index all of the revisions and all of the current code using
Ruby/Odeum then you can search for the actual code and simply ask where
it is in the revisions. So rather than saying, "What happened to file
blah.h" you would say, "Where did that function in blah.h go."

I'm still working out the indexing and querying part, but it mostly
works right now in the code sitting on my computer. Here's an example
usage:

  fcst find -r "Runner main zshaw"

Would print out all the revisions with the words 'Runner' 'main' and
'zshaw' in them, and the files/meta-data mentioning them as well. So,
if I moved the main method to of the Runner class out, you'd get the log
messages saying this, the revisions where this happened, and the files
they moved to.

In addition this also gives you free indexing of your source and I've
added a bunch of very nice source searching files to the find command.

Anyway, that's the really long and unclear idea.

Zed

···

On Wed, 2005-06-29 at 05:42 +0900, Michael Neumann wrote:

Zed A. Shaw wrote:
> <snip>
> * I'm currently polishing off the Ruby/Odeum code so that I can use it
> in FastCST for full text search over code and revisions. This should
> replace just about all the tagging and "revisions follow moves" stuff
> out there. My recent tests show that this approach just rocks.
> <snip>

Could you explain more detailed what you mean with: ...tagging and
"revisions follow moves"?

Could you give a practical example (e.g. how it would work on the
command line), so that I can understand it? :wink:

Michael Neumann wrote:

Daniel Berger wrote:

Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

I'd rather Python won this race, if it meant we could get you to deal
with the 11 bugs, 4 feature requests and 3 patches on the Ruby DBI
project.

Sorry, but I'm no more actively maintaining Ruby/DBI. I should have better announced this before. Anyone, who is still using Ruby/DBI, would like to overtake the project?

Regards,

   Michael

I think I'm going to cry.

Dan

This is really rough, but I got a demo of it up and running. Basically,
I have a small set of code that when it fires up uses Linda as kind of a
poor man's Bonjour/Rendevous. The thought is that, rather than tell
FastCST where the repository for Joe is located, you just start it up
and Joe magically pops up as available.

This solves a major problem with your typical decentralized RCS.
Decentralized RCS systems are nice, but people eventually degenerate to
putting the code on a central system anyway so everyone can find them.
By having a "auto find/zero conf" system in FastCST you can basically
collaborate on the code without having to centralize anything.

At least,that's the idea.

Zed

···

On Wed, 2005-06-29 at 05:45 +0900, Joel VanderWerf wrote:

Michael Neumann wrote:
>
>> * I'm working on the Linda distributed stuff so that I can include
>> zero-config setup.
>>
Also, the use of Linda for zero config setup sounds interesting. Care to
elaborate a bit, or should we wait for the release?

Zed A. Shaw wrote:

<snip FastCST Ruby/Odeum stuff>

Wow. That's cool. Really. I said that out loud when I first read it, even though nobody was in the room.

Doesn't seem, to me, to obviate the need (well, desire) to make mv a special operation in your RCS. For example:
1)It would be nice to view the entire history of a file in one view, but more importantly,
2)How do you solve merging? This is obviously a corner case, but say:
- Dijon refactors a couple of the methods in the Person class into a couple of smaller methods sharing one private method.
- Hellman renames the Person class to User (and renames the file to user.rb), and modifies all the client code.

Sorry, not sure why I have mayo on the brain.

Devin
Hey, at least I didn't make a Kraftwerk reference...

Daniel Berger wrote:

Michael Neumann wrote:

Daniel Berger wrote:

Michael Neumann wrote:

Hi all,

Is there anyone interested in joining a project that aims at creating a
Monotone (www.venge.net/monotone) inspired distributed configuration
management system in Ruby?

Major goal is clean and understandable code, good abstractions and high code
quality, from which further ideas could be implemented more easily than is
the case for Monotone (at least for me).

Ultimate goal of course is the switch from CVS to RCM for the development of
the ruby interpreter :wink:

Or do we want to let Python win the race? :wink:

I'd rather Python won this race, if it meant we could get you to deal
with the 11 bugs, 4 feature requests and 3 patches on the Ruby DBI
project.

Sorry, but I'm no more actively maintaining Ruby/DBI. I should have better announced this before. Anyone, who is still using Ruby/DBI, would like to overtake the project?

Regards,

   Michael

I think I'm going to cry.

Great! You're now an admin of the project. But please let me still submit gems of postgres-pr :wink:

Regards,

   Michael