Ruby 1.9.3 bug: #unpack("p") raises exception even with good bytes

ruby-talk
1

In the following code, why does $p1 succeed but $p2 and $p3 both fail? I
cannot figure this out, and it certainly seems like an unusual ruby bug.
Clearly the bytes being unpacked are the same in all three cases.

    $save = "xyz\0"

    $p1 = [$save].pack("p")
    $p2 = $p1.unpack("i").pack("i")
    $p3 = $p1.bytes.map(&:chr).inject(:+)

    puts "#{$p1.encoding} #{$p2.encoding} #{$p3.encoding}"

    3.times do

      begin
        puts $p1.unpack("p").inspect
      rescue
        puts "failed 1"
      else
        puts "passed 1 #{$p1.bytes.to_a.inspect}"
      end

      puts $p2.unpack("p").inspect rescue
        puts "failed 2 #{$p2.bytes.to_a.inspect}"

      puts $p3.unpack("p").inspect rescue
        puts "failed 3 #{$p3.bytes.to_a.inspect}"

    end

Transcript:

    ASCII-8BIT ASCII-8BIT ASCII-8BIT
    ["xyz\x00"]
    passed 1 [80, 232, 9, 2]
    failed 2 [80, 232, 9, 2]
    failed 3 [80, 232, 9, 2]
    ["xyz\x00"]
    passed 1 [80, 232, 9, 2]
    failed 2 [80, 232, 9, 2]
    failed 3 [80, 232, 9, 2]
    ["xyz\x00"]
    passed 1 [80, 232, 9, 2]
    failed 2 [80, 232, 9, 2]
    failed 3 [80, 232, 9, 2]

···

--
Posted via http://www.ruby-forum.com/.

2

Read the source Array#pack("p") and String#unpack("p").

Array#pack("p") return a res, the res associate to ruby inter object.
so copy the string of res or dup or clone res, unpack("p")
raise #<ArgumentError: no associated pointer> . [demo a,b,c]

But look like clone and dup break the associate. [demo d,e]
There maybe a bug.

demo.rb

···

-----------
def a
  puts "\n--a--\n"
  $save = "xyz\0"
  $p1 = [$save].pack("p")
  $p2 = $p1.bytes.map(&:chr).inject(:+)
  p $p1.unpack("p")
  p $p2.unpack("p") rescue p $!
end
def b
  puts "\n--b--\n"
  $save = "xyz\0"
  $p1 = [$save].pack("p")
  p $p1.unpack("p")
  p $p1.clone.unpack("p") rescue p $!
end
def c
  puts "\n--c--\n"
  $save = "xyz\0"
  $p1 = [$save].pack("p")
  p $p1.unpack("p")
  p $p1.dup.unpack("p") rescue p $!
end
def d
  puts "\n--d--\n"
  $save = "xyz\0"
  $p1 = [$save].pack("p")
  p $p1.unpack("p")
  p $p1.clone.unpack("p") rescue p $!
  p $p1.dup.unpack("p") rescue p $!
end
def e
  puts "\n--e--\n"
  $save = "xyz\0"
  $p1 = [$save].pack("p")
  p $p1.unpack("p")
  p $p1.dup.unpack("p") rescue p $!
  p $p1.clone.unpack("p") rescue p $!
end

a(); b(); c(); d(); e()

__END__

--a--
["xyz\u0000"]
#<ArgumentError: no associated pointer>

--b--
["xyz\u0000"]
#<ArgumentError: no associated pointer>

--c--
["xyz\u0000"]
#<ArgumentError: no associated pointer>

--d--
["xyz\u0000"]
#<ArgumentError: no associated pointer>
["xyz\u0000"]

--e--
["xyz\u0000"]
#<ArgumentError: no associated pointer>
["xyz\u0000"]

-----------

2013/9/21 David Taffs <lists@ruby-forum.com>

In the following code, why does $p1 succeed but $p2 and $p3 both fail? I
cannot figure this out, and it certainly seems like an unusual ruby bug.
Clearly the bytes being unpacked are the same in all three cases.

    $save = "xyz\0"

    $p1 = [$save].pack("p")
    $p2 = $p1.unpack("i").pack("i")
    $p3 = $p1.bytes.map(&:chr).inject(:+)

    puts "#{$p1.encoding} #{$p2.encoding} #{$p3.encoding}"

    3.times do

      begin
        puts $p1.unpack("p").inspect
      rescue
        puts "failed 1"
      else
        puts "passed 1 #{$p1.bytes.to_a.inspect}"
      end

      puts $p2.unpack("p").inspect rescue
        puts "failed 2 #{$p2.bytes.to_a.inspect}"

      puts $p3.unpack("p").inspect rescue
        puts "failed 3 #{$p3.bytes.to_a.inspect}"

    end

Transcript:

    ASCII-8BIT ASCII-8BIT ASCII-8BIT
    ["xyz\x00"]
    passed 1 [80, 232, 9, 2]
    failed 2 [80, 232, 9, 2]
    failed 3 [80, 232, 9, 2]
    ["xyz\x00"]
    passed 1 [80, 232, 9, 2]
    failed 2 [80, 232, 9, 2]
    failed 3 [80, 232, 9, 2]
    ["xyz\x00"]
    passed 1 [80, 232, 9, 2]
    failed 2 [80, 232, 9, 2]
    failed 3 [80, 232, 9, 2]

--
Posted via http://www.ruby-forum.com/\.

3

windwiny wrote in post #1122034:

Read the source Array#pack("p") and String#unpack("p").

I'll try to do that, when I get a chance. But code like this can be
tricky - see below.

Array#pack("p") return a res, the res associate to ruby inter object.
so copy the string of res or dup or clone res, unpack("p")
raise #<ArgumentError: no associated pointer> . [demo a,b,c]

But look like clone and dup break the associate. [demo d,e]
There maybe a bug.

I believe there is a bug. Using the VALUE of the pointer to the "x" in
"xyz\0" makes perfect sense as a handle, since that value is guaranteed
unchanging as long as the GC doesn't move stuff around, and as long as
the caller ensures it doesn't get collected (my $save variable).

However, using the address of the object containing the pointer to the
"x" as a handle seems to make no sense -- the address of the "x" should
be able to be freely copied (or, if it isn't, that should be documented
with how [un]pack("p") works).

You might have seen my long-winded (thus appropriately down-rated) other
post on this issue at
http://stackoverflow.com/questions/18908441/bizarre-no-associated-pointer-error-message-on-ruby-unpackp,
since you tried clone and dup. Turns out dup works "reliably", whereas
clone fails once initially (per session, perhaps per parameter) and
afterwards works "reliably". I'm hoping it's more a bug whereby the
other copy methods cause it to fail, rather than an unfortunate bug
where dup and clone ought to crash likewise, but don't :slight_smile:

I know this gets tricky, because there is no way to tell when the
programmer is done with the raw pointer value, e.g. there is no
guarantee of exactly one unpack per pack. This means some considerable
cleverness is required to prevent e.g. hash tables (associating the
pointer to "x" with a descriptor) growing unboundedly.

It could also be done sticking a header in front of the string (in
memory just before the "x"), at the risk of chaos if the programmer
doesn't protect the "xyz\0" against GC.

Thanks very much for your answer and test cases. I probably won't have
any more time anytime soon to do anything more than glance at the code,
but I hope an experienced ruby implementer gets a chance to research
this.

This is actually a serious problem, because many interfaces require
programmer control over just where the actual pointer objects (created
by pack('p')) are located in storage (e.g. Win32Api requires variable
length arrays of structs containing them).

In addition, reading an ARGV-like structure passed back to Ruby requires
doing an unpack('p') with no corresponding pack('p'), in which case the
object which is the value of $p1 never exists.

Alternatively, the ARGV-like interface could require preallocation on
the Ruby side of the array of pointers, but there's no possible way to
cause pack('p') to allocate its 4-byte strings (containing the pointer
values) adjacently.

Best Regards

···

--
Posted via http://www.ruby-forum.com/\.