Problems with Marshal

Ok, I’ve narrowed it down partially to a newline issue with Marshal. Here
are a sample client and server to demonstrate. Just start the server in one
terminal and the client in another. Uncomment the appropriate lines to view
the different behavior. More comments at end…

SAMPLE SERVER

require ‘socket’
a = [“one”,“two”,“three”] # → This gets wonky results
#a = [“one”,“two”,“three\n”] # → This works
#a.push(“\n”) # → This causes “marshal data too short”
x = Marshal.dump(a)

tcpServer = TCPServer.open(‘localhost’,8877)
$tcpServer = tcpServer

addr = tcpServer.addr
addr.shift # remove “AF_INET”

#puts "Server is on port: " + addr.shift.to_s

while true
Thread.start(tcpServer.accept) do |s|
puts “Accepted connection: #{s}”
r = s.gets
puts “Got #{r} from client”
s.puts(x)
s.close
end
end

SAMPLE CLIENT

require ‘socket’

s = TCPSocket.open(‘localhost’,8877)

s.puts(“hello”) # send whatever
v = s.gets

a = Marshal.load(v)
a.each do |x|
puts x
end
s.close

more comments

The question I have, then, is how do I make sure that the marshalled object
is properly newline terminated?

Regards,

Dan

···

-----Original Message-----
From: Hugh Sasse Staff Elec Eng [mailto:hgs@dmu.ac.uk]

On Wed, 14 Aug 2002, Daniel Berger wrote:

“incompatible marshal file format (can’t be read) (TypeError)”

How are you sending and receiving the data?

Ok, I've narrowed it down partially to a newline issue with Marshal. Here
are a sample client and server to demonstrate. Just start the server in one
terminal and the client in another. Uncomment the appropriate lines to view
the different behavior. More comments at end...

use #read and #write rather than #puts #gets

[...]

while true
   Thread.start(tcpServer.accept) do |s|
      puts "Accepted connection: #{s}"
      r = s.gets
      puts "Got #{r} from client"
      s.puts(x)

         s.write([x.size].pack("N") + x)

      s.close
   end
end

[...]

v = s.gets

   v = s.read(s.read(4).unpack("N")[0])

You can also verify that the string read has the good size

Guy Decoux

The problem is that you’re using gets and puts, which aren’t appropriate
for this kind of data, e.g:

irb(main):004:0> Marshal.dump([“hello”, “world\n”, “\n”])
“\004\006[\010"\nhello"\vworld\n"\006\n”
irb(main):005:0> puts Marshal.dump([“hello”, “world\n”, “\n”])
"
hello"world
"

You can imagine the problem doing a gets on a socket containing that
last string, you aren’t getting your marshalled data back :slight_smile:
You need to be reading all the data the client sends you, not line-wise.

Anyway, isn’t there DRuby - designed specifically for this kind of task?

Tom.

···

Ok, I’ve narrowed it down partially to a newline issue with Marshal. Here
are a sample client and server to demonstrate. Just start the server in one
terminal and the client in another. Uncomment the appropriate lines to view
the different behavior. More comments at end…

SAMPLE SERVER

require ‘socket’
a = [“one”,“two”,“three”] # → This gets wonky results
#a = [“one”,“two”,“three\n”] # → This works
#a.push(“\n”) # → This causes “marshal data too short”
x = Marshal.dump(a)

tcpServer = TCPServer.open(‘localhost’,8877)
$tcpServer = tcpServer

addr = tcpServer.addr
addr.shift # remove “AF_INET”

#puts "Server is on port: " + addr.shift.to_s

while true
Thread.start(tcpServer.accept) do |s|
puts “Accepted connection: #{s}”
r = s.gets
puts “Got #{r} from client”
s.puts(x)

v = s.gets

a = Marshal.load(v)


.^. .-------------------------------------------------------.
/V\ | Tom Gilbert, London, England | http://linuxbrit.co.uk |
/( )\ | Open Source/UNIX consultant | tom@linuxbrit.co.uk |
^^-^^ `-------------------------------------------------------’

From: Hugh Sasse Staff Elec Eng [mailto:hgs@dmu.ac.uk]

How are you sending and receiving the data?

Ok, I’ve narrowed it down partially to a newline issue with Marshal. Here

Curious. I didn’t expect marshal to make much use of whitespace…

are a sample client and server to demonstrate. Just start the server in one
[…]
The question I have, then, is how do I make sure that the marshalled object
is properly newline terminated?

When I wanted to do this I took the somewhat paranoid approach that
the only characters which are guaranteed to be passable between
systems are those that make up base64 – which is why base64 was
invented. See RFC989 Section 5.2 for details of this.

#!/usr/local/bin/ruby -w

Routine to encode an object into a string for transmission.

Author: Hugh Sasse

Created: 07-APR-2000

RCS Infromation:

$Revision: 1.3 $

last put in RCS: $Date: 2000-12-12 11:36:17+00 $

ID: $Id: Encode.rb,v 1.3 2000-12-12 11:36:17+00 hgs Exp hgs $

This takes an object, and it uses Marshal to create a string.

It then converts that string to base 64.

Marshal is used even on strings so that the process is reversible.

If this file is executed directly (i.e. not using “require”) then

it performs a basc self test.

class Encoded
include Marshal

def decode
    @decoded
end

def encode
    @encoded
end

private

def initialize(thing, code = true, depth=-1)
    if code == true
    @decoded = thing
        @encoded = [Marshal.dump(@decoded,depth)].pack("m")
    else
    @encoded = thing
        @decoded = Marshal.load(@encoded.unpack("m")[0])
    end
end

end

if FILE == $0

print "Self test\n"

# define an arbitrary class to save.

class Fish
    def swim
        print "#{@name} is swimming\n"
    end
    def initialize(name)
        @name = name
    end
end

eric = Fish.new("Eric")

# Build an arrray of varying things to save...

x = [1,3.4,"hello!",{"this" => "that"},eric]
p(x)
e = Encoded.new(x, true, 20)

# s is the string to decode...
s = e.encode()
p(s.type())
p(s)

# decode it anew. New instance not a copy of the old
f = Encoded.new(s,false)
result = f.decode()
p(result)

p(result[-1].type())

# show that the las item is really of the class we defined.
result[-1].swim()

end

Regards,

Dan

    Hugh
···

On Thu, 15 Aug 2002, Berger, Daniel wrote:

-----Original Message-----