[Ann] Checkpointing Ruby applications (DragonFlyBSD only)

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to checkpoint processes. That means, that you can store a dump of the running application to disk, from which you can resume later on (possibly on another machine). I mention this, because it's especially interesting for Web frameworks like Wee or Borges, which use Continuations and Threads, that are not marshallable (but checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can find the module here:

   http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Regards,

   Michael

Michael Neumann wrote:

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to checkpoint processes. That means, that you can store a dump of the running application to disk, from which you can resume later on (possibly on another machine). I mention this, because it's especially interesting for Web frameworks like Wee or Borges, which use Continuations and Threads, that are not marshallable (but checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can find the module here:

  http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Maybe I'll give a simple example:

   require 'checkpoint'
   Checkpoint.checkpoint('dump')
   p '.'
   Checkpoint.resume('dump')

This will loop infinite and print '.' on the screen.
Or this one:

   $conn = TCPServer....

   Checkpoint.checkpoint('dump') do
      # setup sockets on resume
      $conn = TCPServer.open('localhost', 9999)
   end

   ....

If you resume from "dump" (type 'checkpt -r dump' on command line, or 'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be executed.

Regards,

   Michael

Michael Neumann wrote:

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to checkpoint processes. That means, that you can store a dump of the running application to disk, from which you can resume later on (possibly on another machine). I mention this, because it's especially interesting for Web frameworks like Wee or Borges, which use Continuations and Threads, that are not marshallable (but checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can find the module here:

  http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Regards,

  Michael

I'm glad there are others who appreciate BSDs. What made you interested?

David Ross

···

--
Hazzle free packages for Ruby?
RPA is available from http://www.rubyarchive.org/

Hello Michael,

   Checkpoint.checkpoint('dump') do
      # setup sockets on resume
      $conn = TCPServer.open('localhost', 9999)
   end

If you resume from "dump" (type 'checkpt -r dump' on command line, or
'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be
executed.

Did you any measurements how much time is saved for startup of a
complexer ruby program with lots of requires ? Something like
FreeRide would be nice. I don't have BSD installed so i can't do it myself.

Also why isn't it possible to directly run the checkpoint file from
the shell ?

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's

David Ross wrote:

Michael Neumann wrote:

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to checkpoint processes. That means, that you can store a dump of the running application to disk, from which you can resume later on (possibly on another machine). I mention this, because it's especially interesting for Web frameworks like Wee or Borges, which use Continuations and Threads, that are not marshallable (but checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can find the module here:

  http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Regards,

  Michael

I'm glad there are others who appreciate BSDs. What made you interested?

In BSD in general or in DragonFly? Well, with DragonFly it's a bit like with Ruby. You can get easily responses from it's creator (indeed, it's easier to get a response from Matt than from Matz ;-), and problems are solved mostly very quickly. You know where to asks, community is small and friendly.

But most importantly, I dropped FreeBSD 5.3, as it paniced on Monday 3 times in a row (maybe it's not FreeBSD's fault, but *I* encountered a lot of crashes in the 5.x line). Now, I'll count every crashes of DragonFly. Zero up to now :wink:

And of course the ideas behind DragonFly are IMHO superior. I am happy each time when Matt has removed a lot of code (rewrote it) or made the API much simpler.

Regards,

   Michael

Lothar Scholz wrote:

Hello Michael,

> Checkpoint.checkpoint('dump') do
> # setup sockets on resume
> $conn = TCPServer.open('localhost', 9999)
> end

> If you resume from "dump" (type 'checkpt -r dump' on command line, or
> 'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be
> executed.

Did you any measurements how much time is saved for startup of a
complexer ruby program with lots of requires ? Something like
FreeRide would be nice. I don't have BSD installed so i can't do it myself.

Not yet. I never had timing problems with lots of requires...
If I can compile freeride (FX...) easily I'll report my results.

Also why isn't it possible to directly run the checkpoint file from
the shell ?

It is possible:

   checkpt -r checkpoint-file

Well, it's in reality a core-dump (or so I'm told), and it has to be sent the CKPTEXIT signal, so that it can perform some setup-tasks... and it will return from the sys_checkpoint syscall with a given value. Not sure how this all could be done easily with a directly executable file, without modifying the core-dump to much.

Regards,

   Michael

Michael Neumann wrote:

Lothar Scholz wrote:

Hello Michael,

> Checkpoint.checkpoint('dump') do
> # setup sockets on resume
> $conn = TCPServer.open('localhost', 9999)
> end

> If you resume from "dump" (type 'checkpt -r dump' on command line, or
> 'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be
> executed.

Did you any measurements how much time is saved for startup of a
complexer ruby program with lots of requires ? Something like
FreeRide would be nice. I don't have BSD installed so i can't do it myself.

Not yet. I never had timing problems with lots of requires...
If I can compile freeride (FX...) easily I'll report my results.

It's not possible to restore freeride (or any GUI programm, I guess), as X11 is involved... Sockets!

Regards,

   Michael

Hello Michael,

Michael Neumann wrote:

Lothar Scholz wrote:

Hello Michael,

> Checkpoint.checkpoint('dump') do
> # setup sockets on resume
> $conn = TCPServer.open('localhost', 9999)
> end

> If you resume from "dump" (type 'checkpt -r dump' on command line, or
> 'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block
will be
> executed.

Did you any measurements how much time is saved for startup of a
complexer ruby program with lots of requires ? Something like
FreeRide would be nice. I don't have BSD installed so i can't do it
myself.

Not yet. I never had timing problems with lots of requires...
If I can compile freeride (FX...) easily I'll report my results.

It's not possible to restore freeride (or any GUI programm, I guess), as
X11 is involved... Sockets!

Right, but you can set a checkpoint before connecting to X11.

Could be nice for large, commercial software. Startup time when you
require hundred thousand lines ruby code is not really funny.

Lisp/Smalltalk users know what i'm talking about.

···

--
Best regards, emailto: scholz at scriptolutions dot com
Lothar Scholz http://www.ruby-ide.com
CTO Scriptolutions Ruby, PHP, Python IDE 's