What are the possibilities in ruby for passing information from one process to another? Specifically, I'm thinking of messaging between a parent process and its forked child.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
And some messaging libraries:
- DRb (standard)
- eventmachine (efficient?)
...what else? And, if anyone has the experience, what are the advantages/disadvantages of each in terms of speed, reliability, system resources?
I don't expect UNIXSocket to be unstable. I'm running over 2 years
with using ruby's socket library without problems. Dont forget to
remove the created socket if your process ends
···
Am Wed, 10 Oct 2007 16:11:15 +0900 schrieb Daniel DeLorme <dan-ml@dan42.com>:
What are the possibilities in ruby for passing information from one
process to another? Specifically, I'm thinking of messaging between a
parent process and its forked child.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
And some messaging libraries:
- DRb (standard)
- eventmachine (efficient?)
...what else? And, if anyone has the experience, what are the
advantages/disadvantages of each in terms of speed, reliability,
system resources?
Intramachine messaging with any of the technologies you mentioned will be
pretty lightweight. The usual thing one tries to do with related processes
is to open a pipe or a socketpair in the parent. Then after the fork, just
send data over the sockets.
···
On 10/10/07, Daniel DeLorme <dan-ml@dan42.com> wrote:
What are the possibilities in ruby for passing information from one
process to another? Specifically, I'm thinking of messaging between a
parent process and its forked child.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
And some messaging libraries:
- DRb (standard)
- eventmachine (efficient?)
...what else? And, if anyone has the experience, what are the
advantages/disadvantages of each in terms of speed, reliability, system
resources?
this is, by far, the easiest approach - it addresses exactly your problem by sticking a drb server in a forked child. the child has arrangements made such that it can *never* outlive it's parent:
On Oct 10, 2007, at 1:11 AM, Daniel DeLorme wrote:
What are the possibilities in ruby for passing information from one process to another? Specifically, I'm thinking of messaging between a parent process and its forked child.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
And some messaging libraries:
- DRb (standard)
- eventmachine (efficient?)
...what else? And, if anyone has the experience, what are the advantages/disadvantages of each in terms of speed, reliability, system resources?
Thanks,
Daniel
--
we can deny everything, except that we have the possibility of being better. simply reflect on that.
h.h. the 14th dalai lama
It may be a bit heavyweight for what you want, but there's ActiveMessaging:
···
On Wed, 10 Oct 2007 16:11:15 +0900, Daniel DeLorme wrote:
What are the possibilities in ruby for passing information from one
process to another? Specifically, I'm thinking of messaging between a
parent process and its forked child.
--
Jay Levitt |
Boston, MA | My character doesn't like it when they
Faster: jay at jay dot fm | cry or shout or hit. http://www.jay.fm | - Kristoffer
DRb will cause you to do the least amount of work. You only need to speak ruby, you don't need to do any serializing of data because it all gets handled for you.
···
On Oct 10, 2007, at 24:11 , Daniel DeLorme wrote:
What are the possibilities in ruby for passing information from one process to another? Specifically, I'm thinking of messaging between a parent process and its forked child.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
And some messaging libraries:
- DRb (standard)
- eventmachine (efficient?)
...what else? And, if anyone has the experience, what are the advantages/disadvantages of each in terms of speed, reliability, system resources?
--
Poor workers blame their tools. Good workers build better tools. The
best workers get their tools to do the work for them. -- Syndicate Wars
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
For completeness:
- named pipes (man fifo)
···
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
On 10/10/07, Daniel DeLorme <dan-ml@dan42.com> wrote:
What are the possibilities in ruby for passing information from one
process to another? Specifically, I'm thinking of messaging between a
parent process and its forked child.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
And some messaging libraries:
- DRb (standard)
- eventmachine (efficient?)
...what else? And, if anyone has the experience, what are the
advantages/disadvantages of each in terms of speed, reliability, system
resources?
Intramachine messaging with any of the technologies you mentioned will be
pretty lightweight. The usual thing one tries to do with related processes
is to open a pipe or a socketpair in the parent. Then after the fork, just
send data over the sockets.
There's also a Ruby MPI binding, but I'm not sure how well maintained it is. I had trouble tracking it down on the web. So I'd go with EventMachine, because that *is* well maintained.
What are the possibilities in ruby for passing information from one process to another? Specifically, I'm thinking of messaging between a parent process and its forked child.
shared memory?
Is there a ruby lib to handle shared memory? Also, I can see how it can be used for "passing information", but for messaging between processes wouldn't it require constant polling to check if a message is waiting in the shared memory?
I don't expect UNIXSocket to be unstable. I'm running over 2 years
with using ruby's socket library without problems. Dont forget to
remove the created socket if your process ends
I just vaguely remember reading in some mailing list or blog something along the lines that "unix domain sockets are flaky". I might have misunderstood.
I don't expect UNIXSocket to be unstable. I'm running over 2 years
with using ruby's socket library without problems. Dont forget to
remove the created socket if your process ends
Ditto. I've been using DRb over UNIX sockets extensively for about 2 years, and no problems here. Data volume and rate tend to be small in my case, though.
···
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
Not that it would be useful but I guess that if you used
two different signals (say SIGUSR1 and SIGUSR2) then you've
got a method of communicating a sequence of binary digits
to another process. If you've got a reasonably accurate
way of timing things you could just manage this with a single
signal because the absence of the signal could be detected.
libraries: (after some digging on rubyforge)
- DRb
- Event Machine
- ActiveMessaging
- Slave
- BackgroundDRb
- System V IPC
- POSIXIPC (not released)
- reliable-msg
- MPI Ruby
- stomp / stompserver / stompmessage
- AP4R (Asynchronous Processing for Ruby)
Frankly I'm still not entirely clear on what are the advantages of each, both in terms of features and in terms of performance. I guess I'll just have to write some code to find out...
I would go with a pipe, because it's easy and sounds like it's all that's needed here. I'm an EventMachine fan, but don't underestimate the trivial approach.
James Edward Gray II
···
On Oct 10, 2007, at 9:39 AM, M. Edward (Ed) Borasky wrote:
So I'd go with EventMachine, because that *is* well maintained.
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
Not that it would be useful but I guess that if you used
two different signals (say SIGUSR1 and SIGUSR2) then you've
got a method of communicating a sequence of binary digits
to another process. If you've got a reasonably accurate
way of timing things you could just manage this with a single
signal because the absence of the signal could be detected.
I smell a ruby quiz: implement drb messaging on top of sigusr[12].
···
--
vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
There's some cleverness to this idea but I would avoid it. Signals interact
very badly with threads and other system facilities, they're not
deterministic, they have serious platform dependencies among different
Unixes, they're heavyweight (resource intensive), and they have difficult
APIs.
The OP simply wants to pass data back and forth between a process and its
forked child. Socketpair or pipe is the orthodox (and the simplest) way to
do this.
···
On 10/10/07, Gary Wright <gwtmp01@mac.com> wrote:
Daniel DeLorme wrote:
> Right now I can think of some messaging primitives:
> - TCPServer/TCPSocket
> - UNIXServer/UNIXSocket (unstable?)
> - IO.pipe (doesn't need port#)
> - Process.kill (impossible to send data)
Not that it would be useful but I guess that if you used
two different signals (say SIGUSR1 and SIGUSR2) then you've
got a method of communicating a sequence of binary digits
to another process. If you've got a reasonably accurate
way of timing things you could just manage this with a single
signal because the absence of the signal could be detected.
libraries: (after some digging on rubyforge)
- DRb
- Event Machine
- ActiveMessaging
- Slave
- BackgroundDRb
- System V IPC
- POSIXIPC (not released)
- reliable-msg
- MPI Ruby
- stomp / stompserver / stompmessage
- AP4R (Asynchronous Processing for Ruby)
Frankly I'm still not entirely clear on what are the advantages of each, both in terms of features and in terms of performance. I guess I'll just have to write some code to find out...
Daniel
1. You missed Rinda, a Linda derivative layered on top of DRb.
2. There are basically two ways to do concurrency/parallelism: shared memory and message passing. I'm not sure what the *real* tradeoffs are -- I've pretty much had my brain hammered with "shared memory bad -- message passing good", but there obviously must be *some* counter-arguments, or shared memory wouldn't exist.
3. System V IPC has three components -- message queues, semaphores, and shared memory segments. So ... if you've found a System V IPC library, you've found a shared memory library.
What no one has yet asked you is: what kind of data do you have to pass
between this fork-parent and child, and by what protocol?
Are they simple commands and responses (as in HTTP)? Is there a
state-machine (as in SMTP)? Or is the data-transfer full-duplex without a
protocol?
Are the data-flows extremely large? Exactly what are your performance
requirements?
Do you have one parent and one child? Or one parent and a great many
children?
···
On 10/12/07, Daniel DeLorme <dan-ml@dan42.com> wrote:
Frankly I'm still not entirely clear on what are the advantages of each,
both in terms of features and in terms of performance. I guess I'll just
have to write some code to find out...
So I'd go with EventMachine, because that *is* well maintained.
I would go with a pipe, because it's easy and sounds like it's all that's needed here. I'm an EventMachine fan, but don't underestimate the trivial approach.
James Edward Gray II
Well, if the architecture is *always* going to be "parent-forked child message passing on a single machine", sure. But what if the task grows beyond the capabilities of that architecture? Remember, I'm one of those people who's in *favor* of "premature" optimization.
But seriously, Ruby has so many user-friendly ways to do concurrency both inside a single machine and across machines that I wouldn't limit myself to parent-forked child message passing on a single machine. I'd use EventMachine because it's more flexible. And to the person who suggested "shared memory", I'd say simply, "Bah!"
···
On Oct 10, 2007, at 9:39 AM, M. Edward (Ed) Borasky wrote:
Hmm. It would be difficult. Since signals are not reliable
in general, you would probably first have to construct the
equivalent of TCP/IP across signals. Drb messaging should be
easy after that...
Gary Wright
···
On Oct 10, 2007, at 5:03 PM, Joel VanderWerf wrote:
Gary Wright wrote:
Daniel DeLorme wrote:
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
Not that it would be useful but I guess that if you used
two different signals (say SIGUSR1 and SIGUSR2) then you've
got a method of communicating a sequence of binary digits
to another process. If you've got a reasonably accurate
way of timing things you could just manage this with a single
signal because the absence of the signal could be detected.
I smell a ruby quiz: implement drb messaging on top of sigusr[12].
There is a whole area of computer security though, covert
signaling methods, where low bandwidth, unreliable communication paths
are still of interest.
···
On Oct 10, 2007, at 6:04 PM, Francis Cianfrocca wrote:
On 10/10/07, Gary Wright <gwtmp01@mac.com> wrote:
Daniel DeLorme wrote:
Right now I can think of some messaging primitives:
- TCPServer/TCPSocket
- UNIXServer/UNIXSocket (unstable?)
- IO.pipe (doesn't need port#)
- Process.kill (impossible to send data)
Not that it would be useful but I guess that if you used
two different signals (say SIGUSR1 and SIGUSR2) then you've
got a method of communicating a sequence of binary digits
to another process. If you've got a reasonably accurate
way of timing things you could just manage this with a single
signal because the absence of the signal could be detected.
There's some cleverness to this idea but I would avoid it. Signals interact
very badly with threads and other system facilities, they're not
deterministic, they have serious platform dependencies among different
Unixes, they're heavyweight (resource intensive), and they have difficult
APIs.
