# Binary file modification
**URL:** https://rubytalk.org/t/binary-file-modification/32193
**Category:** ruby-talk
**Created:** [23 October 2006 02:41 UTC](https://rubytalk.org/t/binary-file-modification/32193 "2006-10-23T02:41:58Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@Rob\_Lee](https://rubytalk.org/u/Rob_Lee)
#### Post date: [23 October 2006 02:41 UTC](https://rubytalk.org/t/binary-file-modification/32193/1 "2006-10-23T02:41:58Z")
Hi
I've been modifying a binary file that contains various data including
some audio. I'm trying to add my own audio (instead of the audio in the
file) which I believe I have done but I need to modify the
content-length header in the file which indicates the length of the
audio sample. I've been reading the content-length data from the file
using something similar to :
f=open(config[:file],"rb")
f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET
length=f.read(3).unpack("H2H2H2").hex.to\_i
f.close
=\> an integer
This basically opens the file as a binary file, skips to the audio
content header data position and then unpacks 3 bytes into a string
(formatted as hex) which is then converted to an integer value.
I'd like to be able to reverse this process and take any an integer
value (1024 in the case shown below) and write it to a binary file with
some header and footer data - something like :
mydata = SOMEHEADERDATA
mydata += ["1024"].pack("someformat")
mydata += AUDIODATA
f=open(config[:file],"wb")
f.write(mydata)
f.close
However I'm a bit stuck on how to pack the data (if this is the correct
solution). If someone could point me in the correct direction it would
be much appreciated.
> **···**
>
> --
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).
---
### Author:  [@Ben\_Nagy](https://rubytalk.org/u/Ben_Nagy)
#### Post date: [23 October 2006 06:37 UTC](https://rubytalk.org/t/binary-file-modification/32193/2 "2006-10-23T06:37:41Z")
> From: list-bounce@example.com
> [mailto:list-bounce@example.com] On Behalf Of Rob Lee
[...]
> I've been reading the content-length data from
> the file
> using something similar to :
>
> f=open(config[:file],"rb")
> f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET
> length=f.read(3).unpack("H2H2H2").hex.to\_i
That should die because Array#hex doesn't exist, but I get the idea. If you
do this kind of thing a lot with different kinds of data then you should
note that to\_i and to\_s both take optional base arguments, so
foo.unpack('H\*').first.to\_i(16) should work
[...]
> I'd like to be able to reverse this process and take any an integer
> value (1024 in the case shown below) and write it to a binary
> file with
> some header and footer data - something like :
[...]
> mydata += ["1024"].pack("someformat")
[...]
> However I'm a bit stuck on how to pack the data (if this is
> the correct
> solution).
Here's the problem - if you need exactly three bytes then you are going to
have to apply your own padding, since the pack routines will only pack
directly as a long or a short which will mostly be 4 and 2 bytes - both of
which could cause you problems. My hacks always involve \<\<'ing a single byte
integer onto a string. In your case, here is a horrible oneliner which you
should not use because it is gross.
num=1024
num.to\_s(16)[0..2].instance\_eval {(self.reverse + '0' \* (6 -
self.length)).reverse}.scan(/../).inject('') {|s,byte| s \<\< byte.to\_i(16)}
You could also try googling BitStruct, which is a ruby library that might
help by defining these headers and footers as structure objects.
Cheers,
ben
> **···**
>
> > -----Original Message-----
---
### Author:  [@Joel\_VanderWerf1](https://rubytalk.org/u/Joel_VanderWerf1)
#### Post date: [23 October 2006 07:54 UTC](https://rubytalk.org/t/binary-file-modification/32193/3 "2006-10-23T07:54:35Z")
Rob Lee wrote:
> Hi
>
> I've been modifying a binary file that contains various data including some audio. I'm trying to add my own audio (instead of the audio in the file) which I believe I have done but I need to modify the content-length header in the file which indicates the length of the audio sample. I've been reading the content-length data from the file using something similar to :
>
> f=open(config[:file],"rb")
> f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET
> length=f.read(3).unpack("H2H2H2").hex.to\_i
> f.close
> =\> an integer
>
> This basically opens the file as a binary file, skips to the audio content header data position and then unpacks 3 bytes into a string (formatted as hex) which is then converted to an integer value.
>
> I'd like to be able to reverse this process and take any an integer value (1024 in the case shown below) and write it to a binary file with some header and footer data - something like :
>
> mydata = SOMEHEADERDATA
> mydata += ["1024"].pack("someformat")
> mydata += AUDIODATA
> f=open(config[:file],"wb")
> f.write(mydata)
> f.close
>
> However I'm a bit stuck on how to pack the data (if this is the correct solution). If someone could point me in the correct direction it would be much appreciated.
Ben's right, you can use BitStruct for this:
require 'bit-struct'
class AudioData \< BitStruct
unsigned :audio\_length, 3\*8, :endian =\> :little
rest :data
# Note: don't use :length as the name of a field, because it will
# conflict with the #length method inherited from String.
# the :endian option can also be :big, :network (== :big), or :native
end
audio\_data = AudioData.new
data = "foo bar baz"
audio\_data.data = data
audio\_data.audio\_length = data.length
p audio\_data
p audio\_data.to\_s
\_\_END\_\_
Output:
#\
"\v\000\000foo bar baz"
> **···**
>
> --
> vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
---
### Author:  [@Hugh\_Sasse](https://rubytalk.org/u/Hugh_Sasse)
#### Post date: [23 October 2006 09:48 UTC](https://rubytalk.org/t/binary-file-modification/32193/4 "2006-10-23T09:48:32Z")
You've had other answers that answer the question as put. Another
approach that may be useful is to look at SNG, a textual means of
accessing the contents of PNG graphics:
[http://www.faqs.org/docs/artu/ch06s01.html#id2910193](http://www.faqs.org/docs/artu/ch06s01.html#id2910193)
HTH
Hugh
> **···**
>
> On Mon, 23 Oct 2006, Rob Lee wrote:
>
> > Hi
> >
> > I've been modifying a binary file that contains various data including
> > some audio. I'm trying to add my own audio (instead of the audio in the
> > file) which I believe I have done but I need to modify the
---
### Author:  [@Joel\_VanderWerf1](https://rubytalk.org/u/Joel_VanderWerf1)
#### Post date: [23 October 2006 07:58 UTC](https://rubytalk.org/t/binary-file-modification/32193/5 "2006-10-23T07:58:17Z")
Joel VanderWerf wrote:
> Ben's right, you can use BitStruct for this:
Forgot the link:
[http://raa.ruby-lang.org/project/bit-struct/](http://raa.ruby-lang.org/project/bit-struct/)
> **···**
>
> --
> vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
---
### Author:  [@Rob\_Lee](https://rubytalk.org/u/Rob_Lee)
#### Post date: [23 October 2006 11:05 UTC](https://rubytalk.org/t/binary-file-modification/32193/6 "2006-10-23T11:05:52Z")
Ben Nagy wrote:
> > From: list-bounce@example.com
> > [mailto:list-bounce@example.com] On Behalf Of Rob Lee
>
> [...]
>
> > I've been reading the content-length data from
> > the file
> > using something similar to :
> >
> > f=open(config[:file],"rb")
> > f.pos=AUDIO\_CONTENT\_HEADER\_OFFSET
> > length=f.read(3).unpack("H2H2H2").hex.to\_i
>
> That should die because Array#hex doesn't exist, but I get the idea. If
> you
> do this kind of thing a lot with different kinds of data then you should
> note that to\_i and to\_s both take optional base arguments, so
> foo.unpack('H\*').first.to\_i(16) should work
Thanks for pointing that out, it's strange as the code definately works
(it returns correct content lengths repeatedly for different files) but
as you say Array#hex doesn't exist - I'll look into this ...
> [...]
>
> > I'd like to be able to reverse this process and take any an integer
> > value (1024 in the case shown below) and write it to a binary
> > file with
> > some header and footer data - something like :
>
> [...]
>
> > mydata += ["1024"].pack("someformat")
>
> [...]
>
> > However I'm a bit stuck on how to pack the data (if this is
> > the correct
> > solution).
>
> Here's the problem - if you need exactly three bytes then you are going
> to
> have to apply your own padding, since the pack routines will only pack
> directly as a long or a short which will mostly be 4 and 2 bytes - both
> of
> which could cause you problems. My hacks always involve \<\<'ing a single
> byte
> integer onto a string. In your case, here is a horrible oneliner which
> you
> should not use because it is gross.
>
> num=1024
> num.to\_s(16)[0..2].instance\_eval {(self.reverse + '0' \* (6 -
> self.length)).reverse}.scan(/../).inject('') {|s,byte| s \<\<
> byte.to\_i(16)}
I'm intrigued by this, I can see you are repacking the int value into
three bytes but I'm still unclear as to how you would then add this to
the file. I guess it needs to be packed as 3 bytes - my guess was
num=1024
newpacket += [num].pack("3C")
However this doesn't seem to work using the reading scheme above. Could
you tell me where I'm going wrong ?
> You could also try googling BitStruct, which is a ruby library that
> might
> help by defining these headers and footers as structure objects.
I'm looking at using bitstruct for the next version of the project, but
as I've almost got this working now I'd like to avoid a re-write at this
stage if possible !
> Cheers,
>
> ben
Thanks for all the help - you've saved me hours of work already !
> **···**
>
> > > -----Original Message-----
>
> --
> Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C).
---
### Author:  [@Rob\_Lee](https://rubytalk.org/u/Rob_Lee)
#### Post date: [23 October 2006 11:29 UTC](https://rubytalk.org/t/binary-file-modification/32193/7 "2006-10-23T11:29:50Z")
> Thanks for pointing that out, it's strange as the code definately works
> (it returns correct content lengths repeatedly for different files) but
> as you say Array#hex doesn't exist - I'll look into this ...
A quick reply to myself - I've just checked the code and I'm missing an
Array#join call in the above example, so the example wasn't correct ...
> **···**
>
> --
> Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C).
---
### Author:  [@Ben\_Nagy](https://rubytalk.org/u/Ben_Nagy)
#### Post date: [23 October 2006 13:03 UTC](https://rubytalk.org/t/binary-file-modification/32193/8 "2006-10-23T13:03:47Z")
> From: list-bounce@example.com
> [mailto:list-bounce@example.com] On Behalf Of Rob Lee
> Sent: Monday, October 23, 2006 6:06 PM
> To: ruby-talk ML
> Subject: Re: Binary file modification
>
> Ben Nagy wrote:
[...]
> \> num=1024
> \> num.to\_s(16)[0..2].instance\_eval {(self.reverse + '0' \* (6 -
> \> self.length)).reverse}.scan(/../).inject('') {|s,byte| s \<\<
> \> byte.to\_i(16)}
> \>
>
> I'm intrigued by this, I can see you are repacking the int value into
> three bytes but I'm still unclear as to how you would then
> add this to
> the file.
It's already packed. The return value of the long expression will be a 3
byte string which you can insert as you like (print it to the IO handle, \<\<
it to another string etc etc). The accumulator in the inject method is a
string {|str,byte| ... might have been more readable, sorry.
ben
> **···**
>
> > -----Original Message-----
---
### Author:  [@Rob\_Lee](https://rubytalk.org/u/Rob_Lee)
#### Post date: [23 October 2006 13:45 UTC](https://rubytalk.org/t/binary-file-modification/32193/9 "2006-10-23T13:45:56Z")
> It's already packed. The return value of the long expression will be a 3
> byte string which you can insert as you like (print it to the IO handle,
> \<\<
> it to another string etc etc). The accumulator in the inject method is a
> string {|str,byte| ... might have been more readable, sorry.
>
> ben
Sorry I should have studied it a little closer ...
I've tried the following and your re-packing doesn't seem to work :
f = open("testfile","wb")
num=359447
f.write(num.to\_s(16)[0..2].instance\_eval {(self.reverse + '0' \* (6 -
self.length)).reverse}.scan(/../).inject('') {|s,byte| s \<\<
byte.to\_i(16)})
f.close
f = open("testfile","rb")
f.pos=0
puts f.read(3).unpack('H\*').first.to\_i(16)
f.close
=\> 1404
It does seem to work for small values like 1024 but not larger ones. I
know there will be an upper limit to the integer size I can fit into the
three bytes but the 359447 value is one I've taken from an existing file
so I believe this is a valid integer to try and pack into these three
bytes.
Any advice ?
> **···**
>
> --
> Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C).
---
### Author:  [@Rob\_Lee](https://rubytalk.org/u/Rob_Lee)
#### Post date: [24 October 2006 09:30 UTC](https://rubytalk.org/t/binary-file-modification/32193/10 "2006-10-24T09:30:37Z")
> It does seem to work for small values like 1024 but not larger ones. I
> know there will be an upper limit to the integer size I can fit into the
> three bytes but the 359447 value is one I've taken from an existing file
> so I believe this is a valid integer to try and pack into these three
> bytes.
>
> Any advice ?
Some more information on this - the maximum value that can be
successfully packed is 4095
Any advice ?
> **···**
>
> --
> Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C).