Problem with binary files in mswin ruby

ruby-talk
1

Because of a problem that manifested itself when running Madeleine on
Windows machines, we have discovered the following:

File.open(“test.dat”, “w”) {|file|
Marshal.dump("\r\n", file)
}
File.open(“test.dat”, “r”) {|file|
file.eof
Marshal.load(file)
}

ArgumentError: marshal data too short

Note that if you remove the line file.eof, the program works without
error.
If you open the file “rb” (setting it to binmode) the program works
without error.
If you add file.rewind after file.eof, the program works without
error.

Note that Marshal.dump and Marshal.load both change the IO Stream to
binary mode internally;
Ruby source marshal.c:
if (rb_respond_to(port, s_binmode)) {
rb_funcall2(port, s_binmode, 0, 0);
}

The conclusion is that an IO stream cannot sucessfully be changed to
binary mode after it has been tested for EOF (unless reset first with
rewind).

Is this expected behaviour?

(Ruby 1.8.1)

Stephen Sykes

(Credit also to Martin Tampe for work on this problem)

± S.D.Sykes - www.stephensykes.com -+
ruby -e ‘_=2;"||25~!A59(1A~47!K~#-6<.A~".scan(/./){
putc$&>$`?$/:+=$&[0]-?4}’

2

Hi,

···

In message “Problem with binary files in mswin ruby” on 04/03/30, haldane jbshaldane@hotmail.com writes:

The conclusion is that an IO stream cannot sucessfully be changed to
binary mode after it has been tested for EOF (unless reset first with
rewind).

Is this expected behaviour?

eof? operation fills the internal stdio buffer before binary mode is
turned on. In that case, turning on the flag is too late.

						matz.
3

Yukihiro Matsumoto wrote:

···

In message “Problem with binary files in mswin ruby” > on 04/03/30, haldane jbshaldane@hotmail.com writes:

The conclusion is that an IO stream cannot sucessfully be changed to
binary mode after it has been tested for EOF (unless reset first with
rewind).

Is this expected behaviour?

eof? operation fills the internal stdio buffer before binary mode is
turned on. In that case, turning on the flag is too late.

Could the methods in Marshal check for this and raise an exception if
they fail to turn on binary mode? (At least on platforms where it
matters).

/Anders

4

Hi,

···

In message “Re: Problem with binary files in mswin ruby” on 04/03/31, Anders Bengtsson ndrsbngtssn@yahoo.se writes:

Could the methods in Marshal check for this and raise an exception if
they fail to turn on binary mode? (At least on platforms where it
matters).

I’m not sure what you want exactly. Making “binmode” to raise
exception when internal buffer is already filled with text-mode
contents?

						matz.
5

Hi,

Is there a reason why eval( “a = not a”) raises a syntax error
whereas eval( “a = !a”) works as I expected ?

Thanks,

Jean-Hugues Robert

6

Something like that, yes. Currently you can get silent errors when
Marshal fails to set the stream to binary. Without knowing about this
internal feature in Marshal, it is very hard to find these errors.
In the case of our Madeleine library, it worked in three our of four
cases, making the single broken case quite confusing.
Alternatively the code in Marshal that sets the stream to binary could
be removed entirely, which would also make it more “fail-fast”.

/Anders

···

— Yukihiro Matsumoto matz@ruby-lang.org wrote:

In message “Re: Problem with binary files in mswin ruby” > on 04/03/31, Anders Bengtsson ndrsbngtssn@yahoo.se writes:

Could the methods in Marshal check for this and raise an exception
if
they fail to turn on binary mode? (At least on platforms where it
matters).

I’m not sure what you want exactly. Making “binmode” to raise
exception when internal buffer is already filled with text-mode
contents?

=====

Anders Bengtsson ndrsbngtssn@yahoo.se
Stockholm, Sweden

Höstrusk och grå moln - köp en resa till solen på Yahoo! Resor på adressen http://se.docs.yahoo.com/travel/index.html

7

matz@ruby-lang.org (Yukihiro Matsumoto) wrote in message news:1080698181.804826.20335.nullmailer@picachu.netlab.jp…

Could the methods in Marshal check for this and raise an exception if
they fail to turn on binary mode? (At least on platforms where it
matters).

I’m not sure what you want exactly. Making “binmode” to raise
exception when internal buffer is already filled with text-mode
contents?

I think that this would probably be sane behaviour. If you change to
binmode and there are characters in the internal buffer that have not
been read then I think an exception would be logical.

Stephen.
± S.D.Sykes - www.stephensykes.com -+

···

“It’s Arsenal 0, Everton 1, and the longer it stays like that the
more you’ve got to fancy Everton to win.” – John Motson