I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
However, at some point I need to make an array of elements extracted from a MySQL database (server side). I can see the list of UTF-8 Greek chars that is created, it's of class array and looks pretty much as it should. However when I serialized it using either Marshal or YAML and send it to the client, the client exists with this error:
···
##############################################
~/Code/Glucose » ruby client.rb
"\x04\b[\n"
client.rb:41:in `load': marshal data too short (ArgumentError)
##############################################
On server side the Marshalled data looks like this:
##############################################
[user@server ~/Code/Project]$ ruby server.rb
["Ααλή", "Ααρών", "αάτρευτα", "αάτρευτε", "Άαχεν"]
Array
"Marshalled \x04\b[\nI\"\r\xCE\x91\xCE\xB1\xCE\xBB\xCE\xAE\x06:\x06ETI\"\x0F\xCE\x91\xCE\xB1\xCF\x81\xCF\x8E\xCE\xBD\x06;\x00TI\"\x15\xCE\xB1\xCE\xAC\xCF\x84\xCF\x81\xCE\xB5\xCF\x85\xCF\x84\xCE\xB1\x06;\x00TI\"\x15\xCE\xB1\xCE\xAC\xCF\x84\xCF\x81\xCE\xB5\xCF\x85\xCF\x84\xCE\xB5\x06;\x00TI\"\x0F\xCE\x86\xCE\xB1\xCF\x87\xCE\xB5\xCE\xBD\x06;\x00T"
##############################################
What I notice is that connection ends when the '\n' character appears. I think that the issue is me using "@con.gets" method instead of something more appropriate. I tried @con.read but since 'read' method waits for the remote party to send an 'EOF' or close the connection stalls.
Is there any other method I can use instead of gets? I did a 'methods.sort' but I can't see and I don't know how to read about these methods using "ri". 'ri TCPSocket::getc' for example doesn't work.
Any ideas and/or thoughts are welcomed, thanks for your time
Best Regards,
Panagiotis (atmosx) Atmatzidis
email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0x1A7BFEC5
gpg --keyserver pgp.mit.edu --recv-keys 1A7BFEC5
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
...
What I notice is that connection ends when the '\n' character appears. I think that the issue is me using "@con.gets" method instead of something more appropriate. I tried @con.read but since 'read' method waits for the remote party to send an 'EOF' or close the connection stalls.
Is there any other method I can use instead of gets? I did a 'methods.sort' but I can't see and I don't know how to read about these methods using "ri". 'ri TCPSocket::getc' for example doesn't work.
You can try
Marshal.load(conn)
on the client side. The Marshal#load method determines just how many bytes to read.
···
On 06/03/2013 11:47 AM, Panagiotis Atmatzidis wrote:
$stdout.sync=true
class Tcp_Server < GServer
def serve( io )
io.set_encoding 'ASCII-8BIT'
until io.closed?
size=io.recv(4).to_i
data= io.recv(size)
if data.size==size
puts "received : '#{Marshal.load(data).join(", ")}"
else
puts "strange size received #{data.size}"
end
io.send('a')
end
end
end
th=Tcp_Server.new(2244,"localhost")
th.start # Start server
On 3 Ιουν 2013, at 21:24 , Joel VanderWerf <joelvanderwerf@gmail.com> wrote:
On 06/03/2013 11:47 AM, Panagiotis Atmatzidis wrote:
Hello,
I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
...
What I notice is that connection ends when the '\n' character appears. I think that the issue is me using "@con.gets" method instead of something more appropriate. I tried @con.read but since 'read' method waits for the remote party to send an 'EOF' or close the connection stalls.
Is there any other method I can use instead of gets? I did a 'methods.sort' but I can't see and I don't know how to read about these methods using "ri". 'ri TCPSocket::getc' for example doesn't work.
You can try
Marshal.load(conn)
on the client side. The Marshal#load method determines just how many bytes to read.
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
This is where I'd stop and use DRb instead.
My first approach was a Sinatra app + YAML (that worked actually although I didn't like it very much though, so I drop it).
Second approach was DRuby, but you can't have bi-directional communication in DRuby without turning the client into a server, which requires opening another port and this in a real-case scenario won't cut it for me.
TCPServer/Socket seems a lot cleaner + I have the chance of learning one thing or two about how a client/server app works at this level. Since time is not an issue and learning as much as possible on the process is one of the goals, all I need to do (in my case) is ti implement 'MuTex' and handle TCPServer/Socket connection errors in a graceful way and I'm on.
If you look at the format of marshalled objects, there is always a way to determine how many bytes to expect. In the case of fixed length data, there is a type field followed by the data (floats for example). In the case of strings, arrays, hashes, etc. there are type and length fields followed by the items (which themselves will have type and possibly length fields). So #load always knows either the number of bytes to read or the number of times to iterate to read all the items. Recursively build the object tree, and you're done.
However, Marshal#load can't know, until this recursion finishes, the total number of bytes. There is no length field for the whole serialized blob. That's why load(sock.read(n)) doesn't work well with marshal.
Btw, if your objects are just strings, numbers, booleans, hashes, and arrays, then msgpack is a great alternative to marshal. It's not ruby specific, it's fast and compact, and it plays well with non-blocking io.
Why non-blocking io? You'll notice that if you Marshal.load(conn) and conn takes a long time to send all the bytes, then the thread that called #load is blocked. So, just start a thread per connection. That's fine if you only need a few connections at a time. With msgpack's buffering, it's possible to read only the complete objects from the connection, and then the thread is free to go on to another connection and read. (See msgpack doc for examples.) Btw, the yajl JSON lib does something like this too.
Cheers!
···
On 06/03/2013 12:41 PM, Panagiotis Atmatzidis wrote:
On 3 Ιουν 2013, at 21:24 , Joel VanderWerf <joelvanderwerf@gmail.com> wrote:
On 06/03/2013 11:47 AM, Panagiotis Atmatzidis wrote:
Hello,
I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
...
What I notice is that connection ends when the '\n' character appears. I think that the issue is me using "@con.gets" method instead of something more appropriate. I tried @con.read but since 'read' method waits for the remote party to send an 'EOF' or close the connection stalls.
Is there any other method I can use instead of gets? I did a 'methods.sort' but I can't see and I don't know how to read about these methods using "ri". 'ri TCPSocket::getc' for example doesn't work.
You can try
Marshal.load(conn)
on the client side. The Marshal#load method determines just how many bytes to read.
Worked fine! Thank you! I don't get it thought, could you elaborate a bit?
Second approach was DRuby, but you can't have bi-directional communication
in DRuby without turning the client into a server, which requires opening
another port and this in a real-case scenario won't cut it for me.
That depends on how you set it up: if avoiding that other port is so
important you could have a queue on client side which is read from the
server in an endless loop. Whenever the queue is empty the call blocks.
Yeah, I know, ugly workaround.
TCPServer/Socket seems a lot cleaner + I have the chance of learning one
thing or two about how a client/server app works at this level. Since
time is not an issue and learning as much as possible on the process is one
of the goals, all I need to do (in my case) is ti implement 'MuTex' and
handle TCPServer/Socket connection errors in a graceful way and I'm on.
Fair enough. What is your application supposed to do?
Kind regards
robert
···
On Wed, Jun 5, 2013 at 10:54 AM, Panagiotis Atmatzidis <atma@convalesco.org>wrote:
On 2013/06/05, at 17:54, Panagiotis Atmatzidis <atma@convalesco.org> wrote:
Hello Robert,
On 5 Ιουν 2013, at 08:52 , Robert Klemme <shortcutter@googlemail.com> wrote:
On Mon, Jun 3, 2013 at 8:47 PM, Panagiotis Atmatzidis <atma@convalesco.org> wrote:
I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
This is where I'd stop and use DRb instead.
My first approach was a Sinatra app + YAML (that worked actually although I didn't like it very much though, so I drop it).
Second approach was DRuby, but you can't have bi-directional communication in DRuby without turning the client into a server, which requires opening another port and this in a real-case scenario won't cut it for me.
TCPServer/Socket seems a lot cleaner + I have the chance of learning one thing or two about how a client/server app works at this level. Since time is not an issue and learning as much as possible on the process is one of the goals, all I need to do (in my case) is ti implement 'MuTex' and handle TCPServer/Socket connection errors in a graceful way and I'm on.
I'm writing a client/server application using TCPSocket and TCPServer. To serialize the objects at this point I use Marshal, since it comes handy.
...
What I notice is that connection ends when the '\n' character appears. I think that the issue is me using "@con.gets" method instead of something more appropriate. I tried @con.read but since 'read' method waits for the remote party to send an 'EOF' or close the connection stalls.
Is there any other method I can use instead of gets? I did a 'methods.sort' but I can't see and I don't know how to read about these methods using "ri". 'ri TCPSocket::getc' for example doesn't work.
You can try
Marshal.load(conn)
on the client side. The Marshal#load method determines just how many bytes to read.
Worked fine! Thank you! I don't get it thought, could you elaborate a bit?
Sure (sorry to be cryptic!).
If you look at the format of marshalled objects, there is always a way to determine how many bytes to expect. In the case of fixed length data, there is a type field followed by the data (floats for example). In the case of strings, arrays, hashes, etc. there are type and length fields followed by the items (which themselves will have type and possibly length fields). So #load always knows either the number of bytes to read or the number of times to iterate to read all the items. Recursively build the object tree, and you're done.
However, Marshal#load can't know, until this recursion finishes, the total number of bytes. There is no length field for the whole serialized blob. That's why load(sock.read(n)) doesn't work well with marshal.
Btw, if your objects are just strings, numbers, booleans, hashes, and arrays, then msgpack is a great alternative to marshal. It's not ruby specific, it's fast and compact, and it plays well with non-blocking io.
Why non-blocking io? You'll notice that if you Marshal.load(conn) and conn takes a long time to send all the bytes, then the thread that called #load is blocked. So, just start a thread per connection. That's fine if you only need a few connections at a time. With msgpack's buffering, it's possible to read only the complete objects from the connection, and then the thread is free to go on to another connection and read. (See msgpack doc for examples.) Btw, the yajl JSON lib does something like this too.
Cheers!
Thanks for the amazing explanation!
I will try msgpack since my objects are just strings and arrays
Second approach was DRuby, but you can't have bi-directional communication in DRuby without turning the client into a server, which requires opening another port and this in a real-case scenario won't cut it for me.
That depends on how you set it up: if avoiding that other port is so important you could have a queue on client side which is read from the server in an endless loop. Whenever the queue is empty the call blocks. Yeah, I know, ugly workaround.
I spent two days trying to find a *normal way* to return the object to the server. Didn't see that sort of solution nowhere online.
TCPServer/Socket seems a lot cleaner + I have the chance of learning one thing or two about how a client/server app works at this level. Since time is not an issue and learning as much as possible on the process is one of the goals, all I need to do (in my case) is ti implement 'MuTex' and handle TCPServer/Socket connection errors in a graceful way and I'm on.
Fair enough. What is your application supposed to do?
Client auth's to the server, using just an string. If auth successfully client requests an object, performs actions and returns an object with results of these actions. Both objects are arrays (text).
On 5 Ιουν 2013, at 15:45 , SEKI Masatoshi <seki.at.druby.org@gmail.com> wrote:
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
Second approach was DRuby, but you can't have bi-directional
communication in DRuby without turning the client into a server, which
requires opening another port and this in a real-case scenario won't cut it
for me.
That depends on how you set it up: if avoiding that other port is so
important you could have a queue on client side which is read from the
server in an endless loop. Whenever the queue is empty the call blocks.
Yeah, I know, ugly workaround.
I spent two days trying to find a *normal way* to return the object to the
server. Didn't see that sort of solution nowhere online.
Are you saying you need an example to implement that?
TCPServer/Socket seems a lot cleaner + I have the chance of learning one
thing or two about how a client/server app works at this level. Since
time is not an issue and learning as much as possible on the process is one
of the goals, all I need to do (in my case) is ti implement 'MuTex' and
handle TCPServer/Socket connection errors in a graceful way and I'm on.
Fair enough. What is your application supposed to do?
Client auth's to the server, using just an string. If auth successfully
client requests an object, performs actions and returns an object with
results of these actions. Both objects are arrays (text).
And where in that scenario is the server issuing requests towards the
client? I only see
client connects and obtains entry object
client invokes session = entry.authenticate(credentials)
server returns session object (if auth successful)
client invokes data = session.get_whatever_i_need()
client invokes result = data.do_the_work() (locally)
client invokes session.the_result_is(result)
client invokes session.close (optional)
Cheers
robert
···
On Wed, Jun 5, 2013 at 12:56 PM, Panagiotis Atmatzidis <atma@convalesco.org>wrote:
On 5 Ιουν 2013, at 12:08 , Robert Klemme <shortcutter@googlemail.com> > wrote:
On Wed, Jun 5, 2013 at 10:54 AM, Panagiotis Atmatzidis < > atma@convalesco.org> wrote:
Okay, from what I get you implemented a minor version of DRuby using sockets yourself. Smart
One quick question is, using 'drb' gem, how can I get the IP or IP(s) of connected clients?
With sockets is straight forward but with DRuby seems to be tricky.
Panagiotis (atmosx) Atmatzidis
email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0x1A7BFEC5
gpg --keyserver pgp.mit.edu --recv-keys 1A7BFEC5
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
Panagiotis (atmosx) Atmatzidis
email: atma@convalesco.org
URL: http://www.convalesco.org
GnuPG ID: 0x1A7BFEC5
gpg --keyserver pgp.mit.edu --recv-keys 1A7BFEC5
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
Second approach was DRuby, but you can't have bi-directional communication in DRuby without turning the client into a server, which requires opening another port and this in a real-case scenario won't cut it for me.
That depends on how you set it up: if avoiding that other port is so important you could have a queue on client side which is read from the server in an endless loop. Whenever the queue is empty the call blocks. Yeah, I know, ugly workaround.
I spent two days trying to find a *normal way* to return the object to the server. Didn't see that sort of solution nowhere online.
Are you saying you need an example to implement that?
TCPServer/Socket seems a lot cleaner + I have the chance of learning one thing or two about how a client/server app works at this level. Since time is not an issue and learning as much as possible on the process is one of the goals, all I need to do (in my case) is ti implement 'MuTex' and handle TCPServer/Socket connection errors in a graceful way and I'm on.
Fair enough. What is your application supposed to do?
Client auth's to the server, using just an string. If auth successfully client requests an object, performs actions and returns an object with results of these actions. Both objects are arrays (text).
And where in that scenario is the server issuing requests towards the client? I only see
client connects and obtains entry object
client invokes session = entry.authenticate(credentials)
server returns session object (if auth successful)
client invokes data = session.get_whatever_i_need()
client invokes result = data.do_the_work() (locally)
client invokes session.the_result_is(result)
client invokes session.close (optional)
On 5 Ιουν 2013, at 14:53 , Robert Klemme <shortcutter@googlemail.com> wrote:
On Wed, Jun 5, 2013 at 12:56 PM, Panagiotis Atmatzidis <atma@convalesco.org> wrote:
On 5 Ιουν 2013, at 12:08 , Robert Klemme <shortcutter@googlemail.com> wrote:
On Wed, Jun 5, 2013 at 10:54 AM, Panagiotis Atmatzidis <atma@convalesco.org> wrote:
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
TCPServer/Socket seems a lot cleaner + I have the chance of learning one
thing or two about how a client/server app works at this level. Since
time is not an issue and learning as much as possible on the process is one
of the goals, all I need to do (in my case) is ti implement 'MuTex' and
handle TCPServer/Socket connection errors in a graceful way and I'm on.
Fair enough. What is your application supposed to do?
Client auth's to the server, using just an string. If auth successfully
client requests an object, performs actions and returns an object with
results of these actions. Both objects are arrays (text).
And where in that scenario is the server issuing requests towards the
client? I only see
client connects and obtains entry object
client invokes session = entry.authenticate(credentials)
server returns session object (if auth successful)
client invokes data = session.get_whatever_i_need()
client invokes result = data.do_the_work() (locally)
client invokes session.the_result_is(result)
Line 7 is superfluous for the client. Oh, and btw. variable "any" in #initialize is pretty useless. Did you mean "@any"?
So the client can dump the results to the remote database? Or pass it to
the server to make the check ups and then dump it to the database?
Of course. Why should it not?
I missed that part if that's the case.
DRb is a complete distributed object communication system. You have value
and reference objects (see DRb::DRbUndumped).
Cheers
robert
···
On Wed, Jun 5, 2013 at 3:27 PM, Panagiotis Atmatzidis <atma@convalesco.org>wrote:
On 5 Ιουν 2013, at 14:53 , Robert Klemme <shortcutter@googlemail.com> > wrote:
On Wed, Jun 5, 2013 at 12:56 PM, Panagiotis Atmatzidis < > atma@convalesco.org> wrote:
On 5 Ιουν 2013, at 12:08 , Robert Klemme <shortcutter@googlemail.com> >> wrote:
Nice thanks, I will need to study a bit SEKI's code which I forked on GH and I'll decide how to proceed. For the time being I'll complete the TCPSocket/Server version but I might use DRb after all.
On 5 Ιουν 2013, at 16:55 , Robert Klemme <shortcutter@googlemail.com> wrote:
--
The wise man said: "Never argue with an idiot. They bring you down to their level and beat you with experience."
if that's the case.