Protocols

ruby-talk
1

Is there a preferred set of ruby libraries for client/server solutions? As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my objectives
to XML. I cannot find a Ruby/Corba implementation. Perl, of course, has
lots of options, but none obviously better than the next. The limitation
seems to be with Ruby supported protocols. I need something which supports
complex data structures. A simple line-terminated packet will not do.

I’ve currently have a working Perl-based program. The program monitors
various logfiles and processes. It correlates errors between processes on
the same machine. The relationships are stored in dbm files using tied
hashes. I’d like to move the data to a server process in the interest of
having all the clients using a central data source rather than their own,
local copy. This should reduce the maintence level-of-effort since only one
data source would need updating rather than a dozen. (Nevermind that
updaing could easily be done via rcp). I’m sure I could use 5.8 thread
features, but why should I try something I know would work?

The point is, I’d like to make a Ruby server. I’ve got a gserver derived
process with dynamic, reloadable methods via a control port. I cannot
figure out a good way for it to swap spit with the perl clients.
Suggestions?

2

“Ray Capozzi” Ray_Capozzi@hotmail.com wrote in message
news:5lRaa.179166$4F3.11146475@news2.east.cox.net…

Is there a preferred set of ruby libraries for client/server solutions?
As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my
objectives
to XML.

Try search the archives for YAML over the past week or two. There is work in
progess using a YAML transport layer on place of Ruby marshalling.

Mikkel

3

Is there a preferred set of ruby libraries for client/server solutions? As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my
objectives to XML. I cannot find a Ruby/Corba implementation. Perl, of
course, has lots of options, but none obviously better than the next. The
limitation seems to be with Ruby supported protocols. I need something
which supports complex data structures. A simple line-terminated packet
will not do.

I’m also interested in this area. A few comments, first.

After dealing with CORBA for a very large project and getting intimately
familiar with it, I seriously dislike it.

Ruby does have some support for marshalling of objects. If Perl support is a
requirement, I would lean towards a bridge implementation which converts
to/from a format that Perl can deal with.

Disclaimer: I haven’t checked out every package in RAA, so there may be
existing libraries that I am unaware of.

I’ve done quite a bit of work setting up some functionality in this area for
Ruby for a project that I am working on. The project is in an alpha state at
the moment, and there is no documentation at all what I have done, but I see
the beginnings of a general purpose framework there. If you decide to
proceed with Ruby, and don’t find an existing library that suits your needs,
let me know.

···

On Sunday 09 March 2003 06:02 pm, Ray Capozzi wrote:

I’ve currently have a working Perl-based program. The program monitors
various logfiles and processes. It correlates errors between processes on
the same machine. The relationships are stored in dbm files using tied
hashes. I’d like to move the data to a server process in the interest of
having all the clients using a central data source rather than their own,
local copy. This should reduce the maintence level-of-effort since only
one data source would need updating rather than a dozen. (Nevermind that
updaing could easily be done via rcp). I’m sure I could use 5.8 thread
features, but why should I try something I know would work?

The point is, I’d like to make a Ruby server. I’ve got a gserver derived
process with dynamic, reloadable methods via a control port. I cannot
figure out a good way for it to swap spit with the perl clients.
Suggestions?

–
Seth Kurtzberg
M. I. S. Corp.
480-661-1849
seth@cql.com

4

Is there a preferred set of ruby libraries for client/server solutions? As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my objectives
to XML.

Assuming you meant to write “objections to XML”, you need not actually see or edit any XML using XML-RPC. It’s just a wire
format. The conversion to/from native data types and icky XML should be hidden inside the client/server code. Of course, if
something goes wrong it can be handy to inspect what’s going over the wire, but your exposure to XML can be kept to a minimum.

James

5

Ray Capozzi wrote:

Is there a preferred set of ruby libraries for client/server solutions? As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my objectives
to XML. I cannot find a Ruby/Corba implementation.

There is Rinn atleast - Rinn download | SourceForge.net

Not sure how complete it is.

···

–
([ Kent Dahl ]/)_ ~ [ http://www.stud.ntnu.no/~kentda/ ]/~
))_student/(( _d L b_/ NTNU - graduate engineering - 5. year )
( __õ|õ// ) )Industrial economics and technological management(
_/ö____/ (_engineering.discipline=Computer::Technology)

6

look out also for the other thread(s):

  • SOAP, DRb and direct method call (was Re: XmlConfigFile usage)

  • DRB and threads

    robert

7

XML-RPC is nice and simple to understand, but it has limitations in its
marshalling of objects (e.g. no aliasing: if a data structure contains two
references to the same object, you will get two copies of the object at the
receiving side). YAML/OKAY and SOAP both don't suffer from this problem, and
have very rich marshalling semantics. If standards-compliance is your main
driver, then SOAP is probably the way to go.

Using either of these approaches will give you beautiful RPC code, in fact
with a little wrapping you can write it can run on either the local or the
remote side wth no changes: e.g.

  #server = MyServ::connect('dbi:oracle:','foo','bar') #for local use
  server = MyServ::connect('druby:192.168.0.1') #for remote use
  data = server.fetch(key)
  ... etc

Just parse the connect string and create an object of the appropriate class.
I have been working on this sort of stuff quite heavily over the last few
days.

But the big caveat: both SOAP and YAML are sloooow. If you're just sending a
couple of strings back and forth they will be fine, but in my case sending a
small collection of objects (about 7K of XML) gave around a 1-second
round-trip time for a single procedure call! The vast majority of this time
is spent just marshalling and unmarshalling objects.

Since I have control of both the back-end and front-end systems, I have
decided to go with DRb for now. It's blindingly fast (I'm getting around 25
RPCs per second, each of which does a DBI query too). This may be just the
excuse you need to rewrite those nasty Perl front-end programs in Ruby :slight_smile:

I didn't compare the speed of XML-RPC because its lack of aliasing means
it's not useful to me, but it may be worth trying. Long term I'd expect it
to die out in favour of SOAP though.

Something to beware of with Ruby implementations of both DRb and SOAP server
(and possibly others) is their interaction with threading. In both cases,
you pass a single object to the server, and it listens for requests: but
each time it answers a request, it start a *new thread* for the method call
but on the *same object* whilst continuing to accept new inbound requests.

For any non-trivial object that means you're going to have to mutex it all
over the place, or write a thread-safe facade object which maintains
(perhaps) a pool of server objects and hands out the calls appropriately.

I cheated - I modified DRb itself.

Regards,

Brian.

···

On Mon, Mar 10, 2003 at 10:02:29AM +0900, Ray Capozzi wrote:

Is there a preferred set of ruby libraries for client/server solutions? As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my objectives
to XML.

8

Wow. Lots of good stuff. No definite winner yet. Kind of sad that in this
day and age this rather simple issue has yet to be resolved. Have all the
smart CS people decided to boycott every-day issues? I am but a mere
accounting guy, so obviously the bribes to look the other way have effected
me.

Cleary I’ve gotta do some due dilligence. I’d like to have a language
independent solution, but as Brain Candler has pointed out, drb has some
real performance advantages. In my place of employment, we deal with C,
C++, TCL, Perl, java, and more. I’d like to not lock into anything so
specific as DRuby. On the other hand, if the gains are so compelling, well
… . .

Well druby seems so cool that I would just have to do it. Too bad that does
not net me anything on the resume. Anyways. . .

I’d like to clarrify my XML concerns. Lucky Stiif makes a point that YAML
is more compact over XML. XML is easy to read. Soo too is Yaml. Xml is
less than compact (more vorbose) than YAML. XML, therefor, seems the lazy
man’s out. I’m not paid to be a script kiddie. Xml means more bytes going
over the wire. YAML simplifies mashelling objects (data). From the old
days, I was taught “Wires are slow. Don’t put stuff on the wire.” Cisco
may disagree. Too be sure, bandwidth is only one measure of total network
performance. Latency is the other. A chatty protocol with small chucks on
a highly latent network will still be slow.

Great set of ideas guys. I’ll have to review the total messages/server to
better understand the chattiness of this scenario. I’ve yet to completely
think though the required throughput for the total system. I know I’ve got
at least 12 servers to start with. More are always potential. In this day
and age, less are equally possible, if you catch the drift.

I surely appreciate the input. It is nice to know that my issues are not
unique. I promise an update as to the course of (in)action. I’m sure we
all appreciate the fogginess of the prior statement. Shoot. Maybe I’ll
take the easy way out. Maybe I’ll just rewrite in VB.

9

“Ray Capozzi” Ray_Capozzi@hotmail.com wrote in message
news:5lRaa.179166$4F3.11146475@news2.east.cox.net…

Is there a preferred set of ruby libraries for client/server solutions?
As
an example, RPC. The protocol needs to exist for Ruby and Perl, so druby
wont work. XML-RPC seems to be the leading candidate, despite my
objectives
to XML.

Try search the archives for YAML over the past week or two. There is work in
progess using a YAML transport layer on place of Ruby marshalling.

What would using YAML-RPC instead of XML-RPC buy you? Is it faster? More stable? Are there more implementations in assorted
languages?

I can understand why some people prefer hand-editing YAML over XML, but when used as a wire format the ease-of-editing factor
becomes less important.

James

···

Mikkel

10

Thanks for the suggestion. It looks promising. I’d like to be able to
telnet to the servers port without the need of speaking yaml to the sever
from the telnet session. It seems that I’d have to adopt yaml whole-hog on
the Ruby side. From the docs, the Yaml methods closes the IO channel upon
doc read. I can see two approaches. First, include a tag at the start of a
session to determine a protocol. Alternatively, I could create a specific
control port process just for telnet. Perhaps there is another alternative.
I could modify the Yaml stuff to not close the channel after reading a doc.

Thanks again for the clue-in.

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com wrote in message
news:3e6be668$0$136$edfadb0f@dtext01.news.tele.dk…

“Ray Capozzi” Ray_Capozzi@hotmail.com wrote in message
news:5lRaa.179166$4F3.11146475@news2.east.cox.net…

Is there a preferred set of ruby libraries for client/server solutions?
As
an example, RPC. The protocol needs to exist for Ruby and Perl, so
druby
wont work. XML-RPC seems to be the leading candidate, despite my
objectives
to XML.

Try search the archives for YAML over the past week or two. There is work
in

···

progess using a YAML transport layer on place of Ruby marshalling.

Mikkel

11

Hi,

From: Brian Candler [mailto:B.Candler@pobox.com]
Sent: Tuesday, March 11, 2003 6:08 AM

Something to beware of with Ruby implementations of both DRb
and SOAP server (and possibly others) is their interaction
with threading. In both cases, you pass a single object to
the server, and it listens for requests: but each time it
answers a request, it start a new thread for the method call
but on the same object whilst continuing to accept new
inbound requests.

Regards to soap4r, it depends on server side implementation.
Using soaplet.rb with WEBrick, you can use
SOAPlet#addRequestServant to add ‘Request scope’ object,
SOAPlet#addServant to add ‘Application scope’ object.

No ‘Session scope’ object for now.

dRuby is from ‘Distributed Ruby’, and the author Seki-san
intends to offer users to be able to move in-process Ruby
environment to distributed Ruby environment easily. No one
expect

s = Servant.new
5.times do
res = s.request
end

to do s.dup for each request. So users who want mutex,
ACL, name service, activation and etc which Ruby does not need,
creates these by themself.

Fortunately there are many samples around dRuby.

Rinda::Ring as lightweight Jini, etc.

Regards,
// NaHi

12

jbritt@ruby-doc.org wrote in message
news:NGEDJNFKAGDNDOIPFPBDGEIBDKAA.jbritt@ruby-doc.org…

I can understand why some people prefer hand-editing YAML over XML, but
when used as a wire format the ease-of-editing factor
becomes less important.

I agree with the ease-of.editing factor.
It is about overhead - YAML is more compact and handles more complex
structures more cleanly.
There is also the semi-binary XML format used on the WAP protocol.

But really, I’d like much more focus on ASN.1.
ASN.1 comes in various encoding forms. A study showed that although there
were processing overhead associated with the more compact formats, they were
still faster. Were are talking about shipping a request in a single small
packet that does not get scattered as opposed to wire and memory expensive
formats such as XML.
Problem is that ASN.1 isn’t widely supported outside specific application
protocols such as encryption and multimedia.

Mikkel

13

What would using YAML-RPC instead of XML-RPC buy you? Is it faster? More
stable? Are there more implementations in assorted languages?

The payload is definitely smaller. The parser isn’t faster, though. And the
!okay/rpc spec is still in development.

I can understand why some people prefer hand-editing YAML over XML, but
when used as a wire format the ease-of-editing factor becomes less
important.

I love using YAML as RPC, simply because of the ease-of-editing factor. I can
edit my blog entries as YAML in Vim and then save to the RPC server. I don’t
have to write a GUI or a browser interface.

If you’re talking machine-to-machine then I see no real advantages, except
that YAML can handle more of Ruby’s native types.

_why

···

On Sunday 09 March 2003 07:35 pm, jbritt@ruby-doc.org wrote:

14

IIWY I’d forget telnet and just have a little client program for manually
driving the RPC calls:

require ‘readline’
$defserver = [‘druby://localhost:9000’]
conn = nil

while line = (ARGV.shift || Readline::readline("rpc> ",true))
args = line.split(/\s+/)
cmd = args.shift
begin
case cmd
when ‘connect’
args = $defserver if args ==
require ‘drb’
conn = DRbObject.new(nil, *args) # replace for SOAP, XML-RPC etc
when ‘quit’
break
else
p conn.send(cmd, *args)
end
rescue Exception
puts “Error: #{$!}”
end
end

Unlike telnet, this also gives you the ability to hit up-arrow and edit your
typing :slight_smile: A more sophisticated parser than space-separated would be good,
but otherwise something like the above is working very well for me.

Incidentally, DRb over TCP keeps the connection up for multiple calls. SOAP
and YAML/OKAY should be able to do so as well (using persistent HTTP/1.1
connections) but you may have to tweak either client or server side to
achieve this.

Cheers,

Brian.

···

On Mon, Mar 10, 2003 at 11:43:16AM +0900, Ray Capozzi wrote:

Thanks for the suggestion. It looks promising. I’d like to be able to
telnet to the servers port without the need of speaking yaml to the sever
from the telnet session. It seems that I’d have to adopt yaml whole-hog on
the Ruby side. From the docs, the Yaml methods closes the IO channel upon
doc read. I can see two approaches. First, include a tag at the start of a
session to determine a protocol. Alternatively, I could create a specific
control port process just for telnet. Perhaps there is another alternative.
I could modify the Yaml stuff to not close the channel after reading a doc.

15

dRuby is from ‘Distributed Ruby’, and the author Seki-san
intends to offer users to be able to move in-process Ruby
environment to distributed Ruby environment easily. No one
expect

s = Servant.new
5.times do
res = s.request
end

to do s.dup for each request. So users who want mutex,
ACL, name service, activation and etc which Ruby does not need,
creates these by themself.

I wasn’t expecting it to dup for each request, but I did at first expect the
requests to be serialized. I can see now why DRb’s main_loop starts a fresh
thread for each connection: it lets it go back to accept() to get new
inbound connections. Also, because DRb TCP connections are persistent,
requests can come in on different sockets in an arbitary order. If it
serialized in the way I had imagined, it would have to keep an array of open
sockets and select() on them.

And of course, there are cases where the concurrent method-call behaviour is
desirable. You just have to be aware that it is happening… and that you
can’t just serve up a ‘normal’ object (i.e. with instances variables not
protected by mutexes) without it breaking.

Fortunately there are many samples around dRuby.

Rinda::Ring as lightweight Jini, etc.

I’m afraid Java-specific terminology means nothing to me, that’s one area
I’ve purposely kept clear of :slight_smile:

Regards,

Brian.

···

On Tue, Mar 11, 2003 at 10:03:53AM +0900, NAKAMURA, Hiroshi wrote:

16

Hi,

From: why the lucky stiff
Sent: Tuesday, March 11, 2003 1:18 AM

If you’re talking machine-to-machine then I see no real
advantages, except
that YAML can handle more of Ruby’s native types.

FYI: Related research is at
http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal

See
http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal%3A%3AUT-Result
how xmlrpc4r and yamlrb (and soap4r, etc) marshals/unmarshals
Ruby’s native types.

yamlrb will be able to support Ruby’s native types
as same as soap4r/CVS does if you want. But I heard
that YAML/yamlrb is rethinking its type system and I am
thinking that you won’t support to avoid complexity.

Regards,
// NaHi

17

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com wrote in message news:3e6c9416$0$142$edfadb0f@dtext01.news.tele.dk…

Problem is that ASN.1 isn’t widely supported outside specific application
protocols such as encryption and multimedia.

ASN.1 is used in a lot of other application domains.
For a (non-exhaustive) list, see
http://asn1.elibel.tm.fr/uses/

A list of (free and non-free) ASN.1 tools is also available at:
http://asn1.elibel.tm.fr/links/#tools

O. Dubuisson

18

Hi,

From: NAKAMURA, Hiroshi
Sent: Tuesday, March 11, 2003 10:04 AM

FYI: Related research is at
http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal

See
http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal%3A%3AUT-Result
how xmlrpc4r and yamlrb (and soap4r, etc) marshals/unmarshals
Ruby’s native types.

I added benchmarks.

Suite: http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal%3A%3ABenchmark
Result: http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal%3A%3ABenchmark-Result

Regards,
// NaHi

19

Quoteing asn1@rd.francetelecom.com, on Tue, Mar 18, 2003 at 06:11:03PM +0900:

“MikkelFJ” mikkelfj-anti-spam@bigfoot.com wrote in message news:3e6c9416$0$142$edfadb0f@dtext01.news.tele.dk…

Problem is that ASN.1 isn’t widely supported outside specific application
protocols such as encryption and multimedia.

That’s because its hideous to work with. It has all the problems and
ambiguities of other formats, such as XML, or the IETF text-based
formats, except that when something goes wrong, you’re staring into a
hex dump trying to figure out what went wrong where. And it can’t make
its mind up about how to represent strings, so you get to pick from a
half-dozen encodings. It’s also got a theoretical abstraction between
the ASN.1, and the many ways of encoding it, some of which doesn’t make
sense in the abstraction (try explaining ASN.1 implicit vs. explicit
tagging without describing BER…). Then explain the DTD format for
XML, then try to explain the latest ASN.1 syntaxes…

To which, everybody says “don’t write hand encoders, use a tool”. Yeah,
sure, they’re hopelessly complex, and have huge run-time code and data
overheads.

Anyhow, sorry for the rant, but I’ve spent a year implmenting encoding
of various crypto standards, and I can’t say I’ve learned to like
ASN.1/BER any more with my experience.

XML is at least readable, by comparison. Though I have to say, given the
success rate implementors have getting DER correct (low), the chance
that anybody is going to cannonicalize XML to verify a signature I
estimate as laughable. I still can’t believe they repeated the worst
mistake of the crypto community with ASN.1, belief in “distinguished”
encodings.

Anyhow, sorry for the rant!

Good luck to anybody who has to work with ASN.1,
Sam

···

ASN.1 is used in a lot of other application domains.
For a (non-exhaustive) list, see
http://asn1.elibel.tm.fr/uses/

A list of (free and non-free) ASN.1 tools is also available at:
http://asn1.elibel.tm.fr/links/#tools

O. Dubuisson

20

This is excellent! Beautiful work, NaHi.

_why

···

On Sunday 16 March 2003 08:58 pm, NAKAMURA, Hiroshi wrote:

I added benchmarks.

Suite: http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal%3A%3ABenchmark
Result:
http://rrr.jin.gr.jp/rwiki?cmd=view;name=Marshal%3A%3ABenchmark-Result