Ruby equivalent of htons or htonl

Hi,

I try to port this c code:
*(uint16_t *)(ptr+FSP_OFFSET_KEY)=htons(p->key);
    *(uint16_t *)(ptr+FSP_OFFSET_SEQ)=htons(p->seq);
    *(uint16_t *)(ptr+FSP_OFFSET_LEN)=htons(p->len);
    *(uint32_t *)(ptr+FSP_OFFSET_POS)=htonl(p->pos);

About uint16_t, I have bit-struct library. But htons messed up my
head.

I found this link:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/18681

But that is 2001, maybe right now we have the function in standard
library. Another point is that library is not portable. It will mess
up in big endian machine. I know I can check the type of machine easy
enough in ruby. But.... still..... I feel better if there is a
function who do the checking for me.

Thank you.

akbarhome wrote:

I try to port this c code:
*(uint16_t *)(ptr+FSP_OFFSET_KEY)=htons(p->key);
    *(uint16_t *)(ptr+FSP_OFFSET_SEQ)=htons(p->seq);
    *(uint16_t *)(ptr+FSP_OFFSET_LEN)=htons(p->len);
    *(uint32_t *)(ptr+FSP_OFFSET_POS)=htonl(p->pos);

About uint16_t, I have bit-struct library. But htons messed up my
head.

I found this link:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/18681

We would still do htons() and hotnl() the same way.

Obviously we don't have uint16_t in Ruby, and we do struct a bit
differently; what are you actually trying to do?

Cheers,
Dave

akbarhome <akbarhome@gmail.com> writes:

I try to port this c code:
*(uint16_t *)(ptr+FSP_OFFSET_KEY)=htons(p->key);
    *(uint16_t *)(ptr+FSP_OFFSET_SEQ)=htons(p->seq);
    *(uint16_t *)(ptr+FSP_OFFSET_LEN)=htons(p->len);
    *(uint32_t *)(ptr+FSP_OFFSET_POS)=htonl(p->pos);

About uint16_t, I have bit-struct library. But htons messed up my
head.

You are thinking about too low a level of porting.

Look, what is this code doing? It is packing up a structure, probably
to be sent over the wire. So let's just do that part, okay? Don't
try to separately do the htonl conversion and the structure-packing.
There's a reason the function isn't in the ruby standard library: the
only time it's needed is when you're packing things up anyway, so it's
built into pack and unpack.

Here's a rough translation of what I think you're trying to write:

# A translation of fsp_pkt_write from
# http://csourcesearch.net/package/gftp/2.0.18/gftp-2.0.18/lib/fsplib/fsplib.c

# Note that in ruby we return the new string, and don't worry about
# preallocating a buffer.

# Also, I'd rename this method to something like "fsp_pkt_make" since
# it doesn't really *write* the data to the output, but that's what
# the function is called in C, so...

def fsp_pkt_write(fsp_pkt)

  fsp_string = [fsp_pkt.cmd, 0, fsp_pkt.key,
                fsp_pkt.seq, fsp_pkt.len, fsp_pkt.pos].pack("CCnnnN")

  # I assume that in the ruby version, p.buf contains both the data
  # block and the "extra data" block

  fsp_string += fsp_pkt.buf

  checksum = 0
  fsp_string.each_byte {|i| checksum += i+1}
  # Note: adding 1 above at each byte is equivalent to adding the length
  fsp_string[1] = (checksum & 0xFF)
  return fsp_string

end

Now, wasn't that easier than hauling out bitstruct to get a
line-by-line translation?

But that is 2001, maybe right now we have the function in standard
library. Another point is that library is not portable. It will mess
up in big endian machine.

Really? Do you have evidence of this, that the htonl as defined in
that ruby-talk message won't work properly on a big endian machine?
Perhaps the problem is how you were intending to use bit-struct?

···

--
s=%q( Daniel Martin -- martin@snowplow.org
       puts "s=%q(#{s})",s.to_a.last )
       puts "s=%q(#{s})",s.to_a.last

You are thinking about too low a level of porting.

Look, what is this code doing? It is packing up a structure, probably
to be sent over the wire. So let's just do that part, okay? Don't
try to separately do the htonl conversion and the structure-packing.
There's a reason the function isn't in the ruby standard library: the
only time it's needed is when you're packing things up anyway, so it's
built into pack and unpack.

Here's a rough translation of what I think you're trying to write:

# A translation of fsp_pkt_write from
#http://csourcesearch.net/package/gftp/2.0.18/gftp-2.0.18/lib/fsplib/f\.\.\.

# Note that in ruby we return the new string, and don't worry about
# preallocating a buffer.

# Also, I'd rename this method to something like "fsp_pkt_make" since
# it doesn't really *write* the data to the output, but that's what
# the function is called in C, so...

def fsp_pkt_write(fsp_pkt)

  fsp_string = [fsp_pkt.cmd, 0, fsp_pkt.key,
                fsp_pkt.seq, fsp_pkt.len, fsp_pkt.pos].pack("CCnnnN")

  # I assume that in the ruby version, p.buf contains both the data
  # block and the "extra data" block

  fsp_string += fsp_pkt.buf

  checksum = 0
  fsp_string.each_byte {|i| checksum += i+1}
  # Note: adding 1 above at each byte is equivalent to adding the length
  fsp_string[1] = (checksum & 0xFF)
  return fsp_string

end

Now, wasn't that easier than hauling out bitstruct to get a
line-by-line translation?

That gives me insight.

Really? Do you have evidence of this, that the htonl as defined in
that ruby-talk message won't work properly on a big endian machine?
Perhaps the problem is how you were intending to use bit-struct?

Actually, big endian machines don't need that htonl function. Here is
the code:
#if defined(BIG_ENDIAN) && !defined(LITTLE_ENDIAN)

#define htons(A) (A)
#define htonl(A) (A)
#define ntohs(A) (A)
#define ntohl(A) (A)

#elif defined(LITTLE_ENDIAN) && !defined(BIG_ENDIAN)

#define htons(A) ((((uint16)(A) & 0xff00) >> 8) | \
                   (((uint16)(A) & 0x00ff) << 8))
#define htonl(A) ((((uint32)(A) & 0xff000000) >> 24) | \
                   (((uint32)(A) & 0x00ff0000) >> 8) | \
                   (((uint32)(A) & 0x0000ff00) << 8) | \
                   (((uint32)(A) & 0x000000ff) << 24))
#define ntohs htons
#define ntohl htohl

#else

#error "Either BIG_ENDIAN or LITTLE_ENDIAN must be #defined, but not
both."

#endif

To make it portable we need to make it like this:
def htonl(h)
  if BIG_ENDIAN_MACHINE then
     h
  else
     do the convert
  end
end

Is there anyway in ruby to detect type of endianness of machine? Thank
you.

···

On Jun 23, 6:12 pm, Daniel Martin <mar...@snowplow.org> wrote:

Daniel Martin wrote:

akbarhome <akbarhome@gmail.com> writes:

I try to port this c code:
*(uint16_t *)(ptr+FSP_OFFSET_KEY)=htons(p->key);
    *(uint16_t *)(ptr+FSP_OFFSET_SEQ)=htons(p->seq);
    *(uint16_t *)(ptr+FSP_OFFSET_LEN)=htons(p->len);
    *(uint32_t *)(ptr+FSP_OFFSET_POS)=htonl(p->pos);

About uint16_t, I have bit-struct library. But htons messed up my
head.

You are thinking about too low a level of porting.

Look, what is this code doing? It is packing up a structure, probably
to be sent over the wire. So let's just do that part, okay? Don't
try to separately do the htonl conversion and the structure-packing.
There's a reason the function isn't in the ruby standard library: the
only time it's needed is when you're packing things up anyway, so it's
built into pack and unpack.

Here's a rough translation of what I think you're trying to write:

# A translation of fsp_pkt_write from # http://csourcesearch.net/package/gftp/2.0.18/gftp-2.0.18/lib/fsplib/fsplib.c

# Note that in ruby we return the new string, and don't worry about
# preallocating a buffer.

# Also, I'd rename this method to something like "fsp_pkt_make" since
# it doesn't really *write* the data to the output, but that's what
# the function is called in C, so...

def fsp_pkt_write(fsp_pkt)

  fsp_string = [fsp_pkt.cmd, 0, fsp_pkt.key,
                fsp_pkt.seq, fsp_pkt.len, fsp_pkt.pos].pack("CCnnnN")

  # I assume that in the ruby version, p.buf contains both the data
  # block and the "extra data" block

  fsp_string += fsp_pkt.buf

  checksum = 0
  fsp_string.each_byte {|i| checksum += i+1}
  # Note: adding 1 above at each byte is equivalent to adding the length
  fsp_string[1] = (checksum & 0xFF)
  return fsp_string

end

Now, wasn't that easier than hauling out bitstruct to get a
line-by-line translation?

bit-struct wouldn't be useful for a line-by-line translation.

The fsp_pkt_write method is nice and simple. (The only problem is that checksum&0xFF is not the same as checksum + (checksum >> 8), but maybe there is something about fsp packets that I'm missing here.)

If you do want to use bit-struct, see below. One disadvantage with bit-struct is that you might be tempted to assign to members after initializing the struct and calculating the checksum, which would invalidate the checksum. Having a single method that does all of the calculations and gives you a string prevents this temptation. YMMV.

require 'bit-struct'

class FSPPacket < BitStruct
   # This is already the default:
   #default_options :endian => :network

   unsigned :cmd, 8
   unsigned :sum, 8

   unsigned :key, 16
   unsigned :seq, 16
   unsigned :len, 16
   unsigned :pos, 32

   rest :buf

   def initialize(*)
     super

     self.len = buf.length # ?? is this right?

     self.sum = 0
     checksum = length
     each_byte { |i| checksum += i }
     self.sum = checksum + (checksum >> 8)
   end
end

fsp = FSPPacket.new do |pkt|
   pkt.cmd = 123
   pkt.key = 42
   pkt.buf = "foo bar"
   # ...
   # all changes within this block will be reflected in the sum and len
end

p fsp # #<FSPPacket cmd=123, sum=91, key=42, seq=0, len=7, pos=0, buf="foo bar">

···

--
       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

pack/unpack should take care of that for you. I whipped up this sample.
Also available from pastie http://pastie.caboo.se/72945

    % cat endian-check.rb
    
    #!/usr/bin/env/ruby
    
    require 'rbconfig'
    include Config

    x = 0xdeadbeef

    endian_type = {
        Array(x).pack("V*") => :little,
        Array(x).pack("N*") => :big
    }

    puts "#{CONFIG['arch']} is a #{endian_type[Array(x).pack("L*")]} endian machine"

And here's some results from various machines

    % ruby endian-check.rb
    i686-darwin8.9.1 is a little endian machine

    % ruby endian-check.rb
    x86_64-openbsd4.0 is a little endian machine

    % ruby endian-check.rb
    powerpc-darwin8.0 is a big endian machine

    % ruby endian-check.rb
    i386-linux is a little endian machine

enjoy

-jeremy

···

On Sat, Jun 23, 2007 at 10:55:06PM +0900, akbarhome wrote:

Is there anyway in ruby to detect type of endianness of machine? Thank
you.

--

Jeremy Hinegardner jeremy@hinegardner.org