# Reading and writing a binary file
**URL:** https://rubytalk.org/t/reading-and-writing-a-binary-file/17074
**Category:** ruby-talk
**Created:** [21 March 2005 13:24 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074 "2005-03-21T13:24:50Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@Chris\_Guenther](https://rubytalk.org/u/Chris_Guenther)
#### Post date: [21 March 2005 13:24 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/1 "2005-03-21T13:24:50Z")
Hi,
I am having trouble reading values from an existing (or maybe not yet
exisiting) binary file:
My target is to open a binary file for reading and writing (if it does not
exisit it should be created).
If the file was not yes existing I'd like to fill it with binary data to a
certain predefined size.
Later on I'd like to position my file position, and read and write whatever
I like.
The following piece does not work as intended:
require 'pp'
UNDER\_UNIX = CONFIG['target\_os'].to\_s.index('linux')!=nil
filename='foo.bin'
size=1024
dont\_write=FileTest.exists?(filename) && File.stat(filename).size==size
begin
fd=File.open(filename,"ab+")
fd.binmode unless UNDER\_UNIX # assuming !UNIX=WINDOWS
rescue =\> e
pp e
fd=File.open(filename,"wb+")
puts "opened #{filename} with 'wib+'"
end
fd.sync=true
fd.seek(0)
unless dont\_write
bin\_data = [0].pack('c')\*(size)
fd.write(bin\_data)
# OK now I really have a file of size 1024 with a binay 0 contents
end
###.... then sometime later using the same file descriptor (fd) :
requestes\_stream\_position = 900
fd.pos=(requestes\_stream\_position)
p "tell=#{fd.tell}"
# Now please read 30 binary 0s into a binary string
bin\_value=fd.read(30)
#### I ASSUMED THAT bin\_value now is a "binary string of size 30", but
unfortunately it is nil !!!
Thanks in adcance,
C.
---
### Author:  [@Robert](https://rubytalk.org/u/Robert)
#### Post date: [21 March 2005 15:14 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/2 "2005-03-21T15:14:54Z")
"Chris Guenther" \ schrieb im Newsbeitrag
news:d1mhu3$bi6$1@news.mch.sbs.de...
> Hi,
>
> I am having trouble reading values from an existing (or maybe not yet
> exisiting) binary file:
> My target is to open a binary file for reading and writing (if it does
not
> exisit it should be created).
> If the file was not yes existing I'd like to fill it with binary data to
a
> certain predefined size.
> Later on I'd like to position my file position, and read and write
whatever
> I like.
>
> The following piece does not work as intended:
>
> require 'pp'
> UNDER\_UNIX = CONFIG['target\_os'].to\_s.index('linux')!=nil
> filename='foo.bin'
> size=1024
> dont\_write=FileTest.exists?(filename) &&
File.stat(filename).size==size
> begin
> fd=File.open(filename,"ab+")
> fd.binmode unless UNDER\_UNIX # assuming !UNIX=WINDOWS
> rescue =\> e
> pp e
> fd=File.open(filename,"wb+")
> puts "opened #{filename} with 'wib+'"
> end
>
> fd.sync=true
> fd.seek(0)
> unless dont\_write
> bin\_data = [0].pack('c')\*(size)
> fd.write(bin\_data)
> # OK now I really have a file of size 1024 with a binay 0 contents
> end
>
> ###.... then sometime later using the same file descriptor (fd) :
> requestes\_stream\_position = 900
> fd.pos=(requestes\_stream\_position)
> p "tell=#{fd.tell}"
> # Now please read 30 binary 0s into a binary string
> bin\_value=fd.read(30)
>
> #### I ASSUMED THAT bin\_value now is a "binary string of size 30",
but
> unfortunately it is nil !!!
>
> Thanks in adcance,
> C.
I don't know what you're doing wrong. Your script works fine for me.
Btw, you don't close the fd.
Here's how I'd do it:
filename='foo.bin'
size=1024
File.open(filename, "ab+") do |fd|
fd.seek(0, IO::SEEK\_END)
if fd.tell \< size
fd.seek 0
fd.write( "\000" \* size );
elsif fd.tell \> size
# if you need the file to be exact this size
fd.truncate size
end
requestes\_stream\_position = 900
fd.seek requestes\_stream\_position
bin\_value = fd.read(30)
p bin\_value
end
Notes:
- don't change binmode depending on platform, always use "b" for binary
files.
- use File.open with block for proper close / cleanup
Kind regards
robert
---
### Author:  [@Chris\_Gunther](https://rubytalk.org/u/Chris_Gunther)
#### Post date: [22 March 2005 11:49 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/3 "2005-03-22T11:49:51Z")
Chris Guenther wrote:
> Hi,
>
> I am having trouble reading values from an existing (or maybe not yet
> exisiting) binary file:
> My target is to open a binary file for reading and writing (if it does not
> exisit it should be created).
> If the file was not yes existing I'd like to fill it with binary data to a
> certain predefined size.
> Later on I'd like to position my file position, and read and write whatever
> I like.
>
> The following piece does not work as intended:
>
> require 'pp'
> UNDER\_UNIX = CONFIG['target\_os'].to\_s.index('linux')!=nil
> filename='foo.bin'
> size=1024
> dont\_write=FileTest.exists?(filename) && File.stat(filename).size==size
> begin
> fd=File.open(filename,"ab+")
> fd.binmode unless UNDER\_UNIX # assuming !UNIX=WINDOWS
> rescue =\> e
> pp e
> fd=File.open(filename,"wb+")
> puts "opened #{filename} with 'wib+'"
> end
>
> fd.sync=true
> fd.seek(0)
> unless dont\_write
> bin\_data = [0].pack('c')\*(size)
> fd.write(bin\_data)
> # OK now I really have a file of size 1024 with a binay 0 contents
> end
>
> ###.... then sometime later using the same file descriptor (fd) :
> requestes\_stream\_position = 900
> fd.pos=(requestes\_stream\_position)
> p "tell=#{fd.tell}"
> # Now please read 30 binary 0s into a binary string
> bin\_value=fd.read(30)
>
> #### I ASSUMED THAT bin\_value now is a "binary string of size 30", but
> unfortunately it is nil !!!
>
> Thanks in adcance,
> C.
Thanks Robert,
you are absolutely right with your remarks using a block. The code I posted was just a small copied fraction of the real code, in which I cannor use a block because the same fd will be stored in a hash for later usage by another method.
Anyway, meanwhile I found the cause of this problem (again you are right, it was not in the code I posted) , but now I have another related problem that tricks my mind. I am trying to patch binary data in a file.
Unfortunately the data I am patching is not the data I get when reading it back from the patched binary file.
Any ideas??
Thanks again in advance,
Chris
#!/usr/bin/env ruby
FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
[?c,?a,?f,?e].pack('cccc')
File.open(FNAME,"ab+") {|fd|
fd.pos=0
bin\_data = [0].pack('c')\*FSIZE
fd.write(bin\_data)
}
File.truncate(FNAME,FSIZE)
File.open(FNAME,"ab+") {|fd|
fd.pos=0
data=fd.read(FSIZE)
data[OFFSET,VALUE.length]=VALUE
fd.pos=0
fd.write(data)
p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
}
File.truncate(FNAME,FSIZE)
File.open(FNAME,"ab+") {|fd|
fd.pos=OFFSET
patched=fd.read(VALUE.length)
p "read modified piece of data = '#{patched}'"
puts "Oooops !!!" unless VALUE==patched
}
---
### Author:  [@Chris\_Gunther](https://rubytalk.org/u/Chris_Gunther)
#### Post date: [22 March 2005 11:29 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/4 "2005-03-22T11:29:51Z")
Thanks Robert,
you are absolutely right with your remarks using a block. The code I posted was just a small copied fraction of the real code, in which I cannor use a block because the same fd will be stored in a hash for later usage by another method.
Anyway, meanwhile I found the cause of this problem (again you are right, it was not in the code I posted) , but now I have another related problem that tricks my mind. I am trying to patch binary data in a file.
Unfortunately the data I am patching is not the data I get when reading it back from the patched binary file.
Any ideas??
Thanks again in advance,
Chris
#!/usr/bin/env ruby
FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
[?c,?a,?f,?e].pack('cccc')
File.open(FNAME,"ab+") {|fd|
fd.pos=0
bin\_data = [0].pack('c')\*FSIZE
fd.write(bin\_data)
}
File.truncate(FNAME,FSIZE)
File.open(FNAME,"ab+") {|fd|
fd.pos=0
data=fd.read(FSIZE)
data[OFFSET,VALUE.length]=VALUE
fd.pos=0
fd.write(data)
p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
}
File.truncate(FNAME,FSIZE)
File.open(FNAME,"ab+") {|fd|
fd.pos=OFFSET
patched=fd.read(VALUE.length)
p "read modified piece of data = '#{patched}'"
puts "Oooops !!!" unless VALUE==patched
}
---
### Author:  [@Robert](https://rubytalk.org/u/Robert)
#### Post date: [22 March 2005 13:59 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/5 "2005-03-22T13:59:53Z")
"Chris Günther" \ schrieb im Newsbeitrag
news:42400070.2050609@gmx.de...
> Thanks Robert,
>
> you are absolutely right with your remarks using a block. The code I
> posted was just a small copied fraction of the real code, in which I
> cannor use a block because the same fd will be stored in a hash for
> later usage by another method.
> Anyway, meanwhile I found the cause of this problem (again you are
> right, it was not in the code I posted) , but now I have another related
> problem that tricks my mind. I am trying to patch binary data in a file.
> Unfortunately the data I am patching is not the data I get when reading
> it back from the patched binary file.
> Any ideas??
>
> Thanks again in advance,
> Chris
>
> #!/usr/bin/env ruby
>
> FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
> [?c,?a,?f,?e].pack('cccc')
>
> File.open(FNAME,"ab+") {|fd|
> fd.pos=0
> bin\_data = [0].pack('c')\*FSIZE
> fd.write(bin\_data)
> }
>
> File.truncate(FNAME,FSIZE)
>
> File.open(FNAME,"ab+") {|fd|
> fd.pos=0
> data=fd.read(FSIZE)
> data[OFFSET,VALUE.length]=VALUE
> fd.pos=0
> fd.write(data)
> p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
> }
You truncate too often:
> File.truncate(FNAME,FSIZE)
> File.open(FNAME,"ab+") {|fd|
> fd.pos=OFFSET
> patched=fd.read(VALUE.length)
> p "read modified piece of data = '#{patched}'"
> puts "Oooops !!!" unless VALUE==patched
> }
Careful when you're copying and pasting.
Regards
robert
---
### Author:  [@Pit](https://rubytalk.org/u/Pit)
#### Post date: [22 March 2005 18:56 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/6 "2005-03-22T18:56:39Z")
Chris Günther schrieb:
> ... I am trying to patch binary data in a file.
> Unfortunately the data I am patching is not the data I get when reading it back from the patched binary file.
> Any ideas??
> ...
> File.open(FNAME,"ab+") {|fd|
> fd.pos=0
> ...
Hi Chris,
it seems that the mode "ab+" is your problem. According to [1], "a+" means open the file for reading and writing, starting at the end of the file. This also means (at least on windows), that the end of file is position 0. So what you are doing above is writing the patched data after the end of the original file.
For patching a binary file I'd change your code to:
FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
[?c,?a,?f,?e].pack('cccc')
# "wb+" for initial data: truncate existing content
File.open(FNAME,"wb+") {|fd|
bin\_data = [0].pack('c')\*FSIZE
fd.write(bin\_data)
}
# "rb+" for the patch: open at the beginning
File.open(FNAME,"rb+") {|fd|
fd.pos=OFFSET
fd.write(VALUE)
p "written modified piece of data = '#{VALUE}'"
}
# "rb" for reading the data
File.open(FNAME,"rb") {|fd|
fd.pos=OFFSET
patched=fd.read(VALUE.length)
p "read modified piece of data = '#{patched}'"
puts "Oooops !!!" unless VALUE==patched
}
Regards,
Pit
[1] [class IO - RDoc Documentation](http://www.ruby-doc.org/core/classes/IO.html)
---
### Author:  [@Carlos](https://rubytalk.org/u/Carlos)
#### Post date: [22 March 2005 19:20 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/7 "2005-03-22T19:20:19Z")
[Chris Günther \, 2005-03-22 12.29 CET]
> Thanks Robert,
>
> you are absolutely right with your remarks using a block. The code I
> posted was just a small copied fraction of the real code, in which I
> cannor use a block because the same fd will be stored in a hash for
> later usage by another method.
> Anyway, meanwhile I found the cause of this problem (again you are
> right, it was not in the code I posted) , but now I have another related
> problem that tricks my mind. I am trying to patch binary data in a file.
> Unfortunately the data I am patching is not the data I get when reading
> it back from the patched binary file.
> Any ideas??
All your problems come from the fact that your are opening the file for
appending ("ab+") instead of for reading ("r+b"). With "ab+", you can set fd.pos=0,
if you want, but it will always write at the end of file.
---
### Author:  [@Chris\_Gunther](https://rubytalk.org/u/Chris_Gunther)
#### Post date: [22 March 2005 17:24 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/8 "2005-03-22T17:24:50Z")
Robert Klemme wrote:
> "Chris Günther" \ schrieb im Newsbeitrag
> news:42400070.2050609@gmx.de...
>
> > Thanks Robert,
> >
> > you are absolutely right with your remarks using a block. The code I
> > posted was just a small copied fraction of the real code, in which I
> > cannor use a block because the same fd will be stored in a hash for
> > later usage by another method.
> > Anyway, meanwhile I found the cause of this problem (again you are
> > right, it was not in the code I posted) , but now I have another related
> > problem that tricks my mind. I am trying to patch binary data in a file.
> > Unfortunately the data I am patching is not the data I get when reading
> > it back from the patched binary file.
> > Any ideas??
> >
> > Thanks again in advance,
> > Chris
> >
> > #!/usr/bin/env ruby
> >
> > FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
> > [?c,?a,?f,?e].pack('cccc')
> >
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=0
> > bin\_data = [0].pack('c')\*FSIZE
> > fd.write(bin\_data)
> > }
> >
> > File.truncate(FNAME,FSIZE)
> >
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=0
> > data=fd.read(FSIZE)
> > data[OFFSET,VALUE.length]=VALUE
> > fd.pos=0
> > fd.write(data)
> > p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
> > }
>
> You truncate too often:
>
> > File.truncate(FNAME,FSIZE)
>
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=OFFSET
> > patched=fd.read(VALUE.length)
> > p "read modified piece of data = '#{patched}'"
> > puts "Oooops !!!" unless VALUE==patched
> > }
>
> Careful when you're copying and pasting.
>
> Regards
>
> robert
I dont see your point. I have to truncate after every write to ensure the file size does not change. Where is the connection to the problem that the read value does not match the written value ?
regards,
C.
---
### Author:  [@Chris\_Gunther](https://rubytalk.org/u/Chris_Gunther)
#### Post date: [22 March 2005 17:24 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/9 "2005-03-22T17:24:51Z")
Robert Klemme wrote:
> "Chris Günther" \ schrieb im Newsbeitrag
> news:42400070.2050609@gmx.de...
>
> > Thanks Robert,
> >
> > you are absolutely right with your remarks using a block. The code I
> > posted was just a small copied fraction of the real code, in which I
> > cannor use a block because the same fd will be stored in a hash for
> > later usage by another method.
> > Anyway, meanwhile I found the cause of this problem (again you are
> > right, it was not in the code I posted) , but now I have another related
> > problem that tricks my mind. I am trying to patch binary data in a file.
> > Unfortunately the data I am patching is not the data I get when reading
> > it back from the patched binary file.
> > Any ideas??
> >
> > Thanks again in advance,
> > Chris
> >
> > #!/usr/bin/env ruby
> >
> > FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
> > [?c,?a,?f,?e].pack('cccc')
> >
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=0
> > bin\_data = [0].pack('c')\*FSIZE
> > fd.write(bin\_data)
> > }
> >
> > File.truncate(FNAME,FSIZE)
> >
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=0
> > data=fd.read(FSIZE)
> > data[OFFSET,VALUE.length]=VALUE
> > fd.pos=0
> > fd.write(data)
> > p "written modified piece of data = '#{data[OFFSET,VALUE.length]}'"
> > }
>
> You truncate too often:
>
> > File.truncate(FNAME,FSIZE)
>
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=OFFSET
> > patched=fd.read(VALUE.length)
> > p "read modified piece of data = '#{patched}'"
> > puts "Oooops !!!" unless VALUE==patched
> > }
>
> Careful when you're copying and pasting.
>
> Regards
>
> robert
I dont see your point. I have to truncate after every write to ensure the file size does not change. Where is the connection to the problem that the read value does not match the written value ?
regards,
C.
---
### Author:  [@Chris\_Gunther](https://rubytalk.org/u/Chris_Gunther)
#### Post date: [23 March 2005 16:49 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/10 "2005-03-23T16:49:48Z")
Pit Capitain wrote:
> Chris Günther schrieb:
>
> > ... I am trying to patch binary data in a file.
> > Unfortunately the data I am patching is not the data I get when reading it back from the patched binary file.
> > Any ideas??
> > ...
> > File.open(FNAME,"ab+") {|fd|
> > fd.pos=0
> > ...
>
> Hi Chris,
>
> it seems that the mode "ab+" is your problem. According to [1], "a+" means open the file for reading and writing, starting at the end of the file. This also means (at least on windows), that the end of file is position 0. So what you are doing above is writing the patched data after the end of the original file.
>
> For patching a binary file I'd change your code to:
>
> FNAME, FSIZE, OFFSET, VALUE = 'foooo.dat', 1024, 32,
> [?c,?a,?f,?e].pack('cccc')
>
> # "wb+" for initial data: truncate existing content
> File.open(FNAME,"wb+") {|fd|
> bin\_data = [0].pack('c')\*FSIZE
> fd.write(bin\_data)
> }
>
> # "rb+" for the patch: open at the beginning
> File.open(FNAME,"rb+") {|fd|
> fd.pos=OFFSET
> fd.write(VALUE)
> p "written modified piece of data = '#{VALUE}'"
> }
>
> # "rb" for reading the data
> File.open(FNAME,"rb") {|fd|
> fd.pos=OFFSET
> patched=fd.read(VALUE.length)
> p "read modified piece of data = '#{patched}'"
> puts "Oooops !!!" unless VALUE==patched
> }
>
> Regards,
> Pit
>
> [1] [class IO - RDoc Documentation](http://www.ruby-doc.org/core/classes/IO.html)
Hi Pit,
thanks for your contribution.
That's exactly the solution I already have. What I dont like about this solution is that you have to open and close file descriptors 3 times just for patching one little file (For sure a working solution but quite some overhead). Just for academical reason it seeems I have to fall back reading the C-sources ☹ to find out what is going wrong under the hood.
Chris
---
### Author:  [@Chris\_Gunther](https://rubytalk.org/u/Chris_Gunther)
#### Post date: [23 March 2005 16:54 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/11 "2005-03-23T16:54:51Z")
Carlos wrote:
> [Chris Günther \, 2005-03-22 12.29 CET]
>
> > Thanks Robert,
> >
> > you are absolutely right with your remarks using a block. The code I posted was just a small copied fraction of the real code, in which I cannor use a block because the same fd will be stored in a hash for later usage by another method.
> > Anyway, meanwhile I found the cause of this problem (again you are right, it was not in the code I posted) , but now I have another related problem that tricks my mind. I am trying to patch binary data in a file.
> > Unfortunately the data I am patching is not the data I get when reading it back from the patched binary file.
> > Any ideas??
>
> All your problems come from the fact that your are opening the file for
> appending ("ab+") instead of for reading ("r+b"). With "ab+", you can set fd.pos=0,
> if you want, but it will always write at the end of file.
Ok, I got it - (maybe only for me) this kind of semanctics is really confusing (and not well described either).
Anyway thanks a lot,
Chris
---
### Author:  [@Robert](https://rubytalk.org/u/Robert)
#### Post date: [23 March 2005 09:09 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/12 "2005-03-23T09:09:52Z")
"Chris Günther" \ schrieb im Newsbeitrag
news:42405416.5050603@gmx.de...
\
> I dont see your point. I have to truncate after every write to ensure
> the file size does not change. Where is the connection to the problem
> that the read value does not match the written value ?
I don't see the point either. :-} Sorry for the confusion, I overlooked
that you used the other form of truncate with a length.
Regards
robert
---
### Author:  [@Pit](https://rubytalk.org/u/Pit)
#### Post date: [23 March 2005 17:30 UTC](https://rubytalk.org/t/reading-and-writing-a-binary-file/17074/13 "2005-03-23T17:30:03Z")
Chris Günther schrieb:
> That's exactly the solution I already have. What I dont like about this solution is that you have to open and close file descriptors 3 times just for patching one little file (For sure a working solution but quite some overhead).
In order to patch an existing file, all you have to do is
> > File.open(FNAME,"rb+") {|fd|
> > fd.pos=OFFSET
> > fd.write(VALUE)
> > p "written modified piece of data = '#{VALUE}'"
> > }
The other File.open calls are for creating the test file and for testing the contents of the patched file. I copied them from your initial script. Of course you could do all three steps (create, patch, test) in a single File.open block.
Regards,
Pit