Unpack signed short in network (big-endian) byte order

Is there any particular reason why there is no format specifier in the
String.unpack method for unpacking a signed short in network byte
order? There seems only to be one for unsigned, which can then be
converted to signed with a proc like this:

length = 16
max = 2**length-1
mid = 2**(length-1)
to_signed = proc {|n| (n >= mid) ? -((n ^ max) + 1) : n}

It seems reasonable to add a format specifier to do this directly.
Thoughts?

baumanj@gmail.com wrote:

Is there any particular reason why there is no format specifier in the
String.unpack method for unpacking a signed short in network byte
order? There seems only to be one for unsigned, which can then be
converted to signed with a proc like this:

length = 16
max = 2**length-1
mid = 2**(length-1)
to_signed = proc {|n| (n >= mid) ? -((n ^ max) + 1) : n}

It seems reasonable to add a format specifier to do this directly.
Thoughts?

I'd vote for that. Looks like you already found this thread :slight_smile:

http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-talk/165774?165771-168359

···

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Perhaps this should be a new subject like "adding formats for
pack/unpack". I gave this a bit of thought, and there are quite a few
formats that seem to be missing. I would naturally assume that there
should be support for each combination of

{unsigned, signed} x {native, network, little-endian} x {short, int,
long}

However, of these 18 formats, 8 are missing. There are more than enough
unused characters to specify them (j J k K o O r R t T W y Y z). I
tried to come up with consistent mappings based on what already exists
and this is what I came up with (?s indicate that the format is not
currently supported):

S unsigned native short
s signed native short
n unsigned network short
o ? signed network short
v unsigned little-endian short
r ? signed little-endian short

I unsigned native int
i signed native int
J ? unsigned network int
j ? signed network int
K ? unsigned little-endian int
k ? signed little-endian int

L unsigned native long
l signed native long
N unsigned network long
y ? signed network long
V unsigned little-endian long
y ? signed little-endian long

I'm thinking of submitting an RCR for this, but before I go through the
effort of figuring out the c code to achieve it, I wanted some opinions
as to whether this is worthwhile and whether the mappings are sensible.

Thanks.

Joel VanderWerf wrote:

···

baumanj@gmail.com wrote:
> Is there any particular reason why there is no format specifier in the
> String.unpack method for unpacking a signed short in network byte
> order? There seems only to be one for unsigned, which can then be
> converted to signed with a proc like this:
>
> length = 16
> max = 2**length-1
> mid = 2**(length-1)
> to_signed = proc {|n| (n >= mid) ? -((n ^ max) + 1) : n}
>
> It seems reasonable to add a format specifier to do this directly.
> Thoughts?
>

I'd vote for that. Looks like you already found this thread :slight_smile:

http://blade.nagaokaut.ac.jp/cgi-bin/vframe.rb/ruby/ruby-talk/165774?165771-168359

--
      vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407

Perhaps this should be a new subject like "adding formats for
pack/unpack". I gave this a bit of thought, and there are quite a few
formats that seem to be missing. I would naturally assume that there
should be support for each combination of

{unsigned, signed} x {native, network, little-endian} x {short, int,
long}

However, of these 18 formats, 8 are missing. There are more than enough
unused characters to specify them (j J k K o O r R t T W y Y z). I
tried to come up with consistent mappings based on what already exists
and this is what I came up with (?s indicate that the format is not
currently supported):

[...]

I'm thinking of submitting an RCR for this,

Don't bother with an RCR, just submit a patch to ruby-core.

but before I go through the
effort of figuring out the c code to achieve it, I wanted some opinions
as to whether this is worthwhile and whether the mappings are sensible.

What does perl or python do for the same types?

···

On Mar 30, 2006, at 1:38 PM, baumanj@gmail.com wrote:

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

i use pack/unpack alot and would love to see more types handled. i'm with
eric on checking perl/python. however, the reality is that pack/unpack are so
dang cryptic it hardly matters - i always need to read the docs :wink: that said
they are extremely useful.

good luck.

-a

···

On Fri, 31 Mar 2006, baumanj@gmail.com wrote:

Perhaps this should be a new subject like "adding formats for
pack/unpack". I gave this a bit of thought, and there are quite a few
formats that seem to be missing. I would naturally assume that there
should be support for each combination of

{unsigned, signed} x {native, network, little-endian} x {short, int,
long}

However, of these 18 formats, 8 are missing. There are more than enough
unused characters to specify them (j J k K o O r R t T W y Y z). I
tried to come up with consistent mappings based on what already exists
and this is what I came up with (?s indicate that the format is not
currently supported):

S unsigned native short
s signed native short
n unsigned network short
o ? signed network short
v unsigned little-endian short
r ? signed little-endian short

I unsigned native int
i signed native int
J ? unsigned network int
j ? signed network int
K ? unsigned little-endian int
k ? signed little-endian int

L unsigned native long
l signed native long
N unsigned network long
y ? signed network long
V unsigned little-endian long
y ? signed little-endian long

I'm thinking of submitting an RCR for this, but before I go through the
effort of figuring out the c code to achieve it, I wanted some opinions
as to whether this is worthwhile and whether the mappings are sensible.

Thanks.

--
share your knowledge. it's a way to achieve immortality.
- h.h. the 14th dali lama

Oops, 'y' can't be both signed network long and signed little-endian
long. How about this:

S unsigned native short
s signed native short
n unsigned network short
o ? signed network short
v unsigned little-endian short
r ? signed little-endian short

I unsigned native int
i signed native int
J ? unsigned network int
j ? signed network int
K ? unsigned little-endian int
k ? signed little-endian int

L unsigned native long
l signed native long
N unsigned network long
O ? signed network long
V unsigned little-endian long
R ? signed little-endian long

Hi,

However, of these 18 formats, 8 are missing. There are more than enough
unused characters to specify them (j J k K o O r R t T W y Y z). I
tried to come up with consistent mappings based on what already exists
and this is what I came up with (?s indicate that the format is not
currently supported):

The pack templates are derived from Perl, and I don't want to add our
own templates. I just talked with Larry Wall who gave presentation at
YAPC::Asia Tokyo, and he said he was not sure how to deal pack
templates in Perl6 yet. Maybe we can meet agreement to add new pack
templates, or we can think of other feature to pack/unpack binary
data, such as cstruct module in Python.

              matz.

···

In message "Re: unpack signed short in network (big-endian) byte order" on Fri, 31 Mar 2006 06:38:43 +0900, "baumanj@gmail.com" <baumanj@gmail.com> writes:

Perl's pack options seem to be largely the same. It has the same holes
that ruby does. One interesting wrinkle is that where ruby uses the
underscore ("_") suffix to enforce native lengths, perl uses a bang
("!"). I don't know why that's different.

http://www.xav.com/perl/lib/Pod/perlfunc.html#item_pack

Python has a rather different approach. There are only a few specifiers
but integer length and byte-order are defined by a modifier at the
front of the format string.

I sort of like the Python idea, because then we wouldn't have to
introduce more cryptic format specifiers. However, it eliminates the
(potentially unlikely) ability to have different byte-ordering for
different values within the array being packed. Perhaps the best
solution is to allow the modifier at the beginning to establish a
default, but allow modifiers after specifiers as well in order to
override the default. On the other hand, maybe it would be best to
stick to one or the other approach. The former is simpler and probably
covers the common usage, but the latter is more similar to the way the
function currently works.

I don't know. Opinions?

Eric Hodel wrote:

···

On Mar 30, 2006, at 1:38 PM, baumanj@gmail.com wrote:

> Perhaps this should be a new subject like "adding formats for
> pack/unpack". I gave this a bit of thought, and there are quite a few
> formats that seem to be missing. I would naturally assume that there
> should be support for each combination of
>
> {unsigned, signed} x {native, network, little-endian} x {short, int,
> long}
>
> However, of these 18 formats, 8 are missing. There are more than
> enough
> unused characters to specify them (j J k K o O r R t T W y Y z). I
> tried to come up with consistent mappings based on what already exists
> and this is what I came up with (?s indicate that the format is not
> currently supported):

[...]

> I'm thinking of submitting an RCR for this,

Don't bother with an RCR, just submit a patch to ruby-core.

> but before I go through the
> effort of figuring out the c code to achieve it, I wanted some
> opinions
> as to whether this is worthwhile and whether the mappings are
> sensible.

What does perl or python do for the same types?

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

Hi,

···

In message "Re: unpack signed short in network (big-endian) byte order" on Fri, 31 Mar 2006 08:08:43 +0900, "baumanj@gmail.com" <baumanj@gmail.com> writes:

Perl's pack options seem to be largely the same. It has the same holes
that ruby does. One interesting wrinkle is that where ruby uses the
underscore ("_") suffix to enforce native lengths, perl uses a bang
("!"). I don't know why that's different.

Since Perl used to use underscore when I implemented it. Besides Ruby
allows "!" as well.

              matz.

Yukihiro Matsumoto wrote:

>Perl's pack options seem to be largely the same. It has the same holes
>that ruby does. One interesting wrinkle is that where ruby uses the
>underscore ("_") suffix to enforce native lengths, perl uses a bang
>("!"). I don't know why that's different.

Since Perl used to use underscore when I implemented it. Besides Ruby
allows "!" as well.

Oh? I don't think that's mentioned in the rdoc for pack/unpack. In any
case, do you have any thoughts about adding these other formats?