64-bit integers in network byte-order

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

···

--
Posted via http://www.ruby-forum.com/.

Hi Francis,

A quick scan of the Ruby source tells me that you'll have to implement
it yourself. Network byte order is big-endian so you should be able to
do it like this:

    [val >> 32, val & 0xFFFFFFFF].pack("NN")

At least I think that is correct.

Cheers,
Dave

···

On 9/22/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

--
Posted via http://www.ruby-forum.com/\.

i don't know the answer, but maybe a hack?

     harp:~ > cat a.rb
     class String
       LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
       BIG_ENDIAN = !LITTLE_ENDIAN

       def hton!() BIG_ENDIAN ? self : replace(reverse) end
       def hton() dup.hton! end
       def noth!() BIG_ENDIAN ? self : replace(reverse) end
       def noth() dup.noth! end
     end

     forty_two = [42].pack('Q')
     p forty_two

     forty_two = [42].pack('Q').hton!
     p forty_two

     harp:~ > ruby a.rb
     "*\000\000\000\000\000\000\000"
     "\000\000\000\000\000\000\000*"

also, maybe joel's bitstruct package

   bit-struct

regards.

-a

···

On Fri, 22 Sep 2006, Francis Cianfrocca wrote:

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei

Thanks, guys. I sort of figured I'd have to hack it. Ara, I thought of
your approach (detecting endian-ness locally) but really hoped I could
avoid it! (Ghostbusters: "I feel so funky.")

Combining your approach with Dave's (and passing both through several
additional processing stages), I end up with:

···

On 9/22/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

#--------------------------------------
BigEndian = [1].pack("s") == [1].pack("n")

def htonq value
  BigEndian ? val : ([val].pack("Q").reverse.unpack("Q").first)
end
#--------------------------------------

and of course ntohq is identical to htonq.

...

      def hton!() BIG_ENDIAN ? self : replace(reverse) end

replace(reverse) is no different from reverse!, is it?

···

ara.t.howard@noaa.gov wrote:

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Sorry, I really can't resist this. Do you know who the big-endians and
the little-endians are in literary history?

···

On 9/22/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:
> LITTLE_ENDIAN = ([42].pack('i')[0] == 42)

       BIG_ENDIAN = !LITTLE_ENDIAN

Funny, I wrote almost exactly the above code just recently. But, it may not be necessary. Depending on your situation, you might consider using the "w" (BER-compressed integer) packing directive instead. It guarantees that arbitrarily long integers are stored in a standard way (big endian). The caveats are (1) it produces variable-length packed strings and (2) it only supports positive integers. However, because BER-compressed integers are self-describing, in the sense that you can determine when you've reached the end of the encoded string, you could get around the first limitation (if it proves to be a limitation for your usage scenario) by left justifying the BER-compressed integer in a fixed field. For my situation, I just changed the definition of my wire protocol to support variable-length "w"-encoded strings and didn't bother left justifying.

  --Young

···

On Sep 22, 2006, at 8:56 AM, Francis Cianfrocca wrote:

On 9/22/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

All,
does Ruby have a way to convert 64-bit integers to and from network
order in a platform-independent way?

For 32-bit ints, I can use [value].pack("N"). The conversion operator
for 64-bit ints ("Q") doesn't have a network-order variant. (I assume
this is because the underlying byte-order converters (htons/htonl) don't
have a 64-bit version on 32-bit platforms.)

Thanks, guys. I sort of figured I'd have to hack it. Ara, I thought of
your approach (detecting endian-ness locally) but really hoped I could
avoid it! (Ghostbusters: "I feel so funky.")

Combining your approach with Dave's (and passing both through several
additional processing stages), I end up with:

#--------------------------------------
BigEndian = [1].pack("s") == [1].pack("n")

def htonq value
BigEndian ? val : ([val].pack("Q").reverse.unpack("Q").first)
end
#--------------------------------------

and of course ntohq is identical to htonq.

it's totally different!

err, i forgot about 'reverse!' :wink:

cheers.

-a

···

On Sat, 23 Sep 2006, Joel VanderWerf wrote:

ara.t.howard@noaa.gov wrote:
...

      def hton!() BIG_ENDIAN ? self : replace(reverse) end

replace(reverse) is no different from reverse!, is it?

--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei

no. do tell...

-a

···

On Sat, 23 Sep 2006, Francis Cianfrocca wrote:

On 9/22/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:

       LITTLE_ENDIAN = ([42].pack('i')[0] == 42)
       BIG_ENDIAN = !LITTLE_ENDIAN

Sorry, I really can't resist this. Do you know who the big-endians and
the little-endians are in literary history?

--
in order to be effective truth must penetrate like an arrow - and that is
likely to hurt. -- wei wu wei

Thanks for the suggestion, but I'm having to interoperate with a wire
protocol (AMQP) that defines certain fields as eight-octet integers in
network-order, so I don't have a choice in the matter. Clearly
BER-encoding has the advantage of permitting nearly-arbitrary size,
but I think what has happened here is that the protocol designers are
heavily Java-centric, and eight-octets-in-network-order is Java's
native encoding for a long int.

···

On 9/22/06, Young Hyun <youngh@caida.org> wrote:
> Funny, I wrote almost exactly the above code just recently. But, it

may not be necessary. Depending on your situation, you might
consider using the "w" (BER-compressed integer) packing directive
instead. It guarantees that arbitrarily long integers are stored in
a standard way (big endian). The caveats are (1) it produces
variable-length packed strings and (2) it only supports positive
integers. However, because BER-compressed integers are self-
describing, in the sense that you can determine when you've reached
the end of the encoded string, you could get around the first
limitation (if it proves to be a limitation for your usage scenario)
by left justifying the BER-compressed integer in a fixed field. For
my situation, I just changed the definition of my wire protocol to
support variable-length "w"-encoded strings and didn't bother left
justifying.

In Gulliver's Travels, a "little-endian" prefers to crack open
soft-boiled eggs from the little end. The little-endians and the
big-endians are unable to settle their dispute, so they go to war. The
story has reacquired its piquancy, now that religious war is back in
vogue.

Sorry for the offtopic, and thanks for your help, Ara.

···

On 9/22/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:
> > Sorry, I really can't resist this. Do you know who the big-endians and

> the little-endians are in literary history?

no. do tell...

Great, great, great can we talk about Alice again :wink:

Really, what a b..... I am today, appologies to all who suffer from my
humor, I gonna pipe down now.

Robert

···

On 9/22/06, Francis Cianfrocca <garbagecat10@gmail.com> wrote:

On 9/22/06, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov> wrote:
> > Sorry, I really can't resist this. Do you know who the big-endians and
> > the little-endians are in literary history?
>
> no. do tell...
>

In Gulliver's Travels, a "little-endian" prefers to crack open
soft-boiled eggs from the little end. The little-endians and the
big-endians are unable to settle their dispute, so they go to war. The
story has reacquired its piquancy, now that religious war is back in
vogue.

Sorry for the offtopic, and thanks for your help, Ara.

--
Deux choses sont infinies : l'univers et la bêtise humaine ; en ce qui
concerne l'univers, je n'en ai pas acquis la certitude absolue.

- Albert Einstein