String doesnt auto dup on modification

ruby-talk
21

Simon Krahnke wrote:

* Mike Gold <mike.gold.4433@gmail.com> (2009-01-23) schrieb:
> String.public_instance_methods(false).grep(/!/).map(&:to_sym) +

Is that a new feature?

Symbol#to_proc is new in ruby core since 1.8.7+. It had been previously
defined by ActiveSupport and, I think, facets.

make the to_sym method act as block. Which to_sym method?

The to_sym method of each item in the array. symbol_to_proc is defined
somewhat like:
lambda {|x, *args| x.send(self, *args)}
So foo.map(&:to_sym) behaves like foo.map {|x| x.send(:to_sym)}

> DESTRUCTIVE_METHODS.each { |m|
> define_method(m) { |*args, &block|
> __setobj__(__getobj__.dup)
> __getobj__.send(m, *args, &block)

WTF? What's this __[gs]etobj__ about?

Those are methods defined by DelegateClass(). They are used to get and set the
object that is delegated to. In this case they cause the proxy-object to
delegate to a copy of the original string instead of the original string
itself when a destructive method is called.

HTH,
Sebastian

···

--
NP: Metallica - Jump In The Fire
Jabber: sepp2k@jabber.org
ICQ: 205544826

22

RK Sentinel wrote:

class Object
   def dupf
     frozen? ? self : dup
   end
end

and then

class MyClass
   def initialize(name)
     @name = name.dupf
   end
end

Kind regards

  robert

thanks, this looks very helpful.

Beware: this may not solve your problem. It will work if the passed-in
object is itself a String, but not if it's some other object which
contains Strings.

Try:

a = ["hello", "world"]
b = a.dupf
b[0] << "XXX"
p a

However, deep-copy is an even less frequently seen solution.

In response to the prev post by Brian, yes its a library for use by
others.

It's hard to provide concrete guidance without knowing what this library
does, but it sounds like it builds some data structure which includes
the string passed in.

I believe that a typical library is behaving correctly if it just stores
a reference to the string.

If a problem arises because the caller is later mutating that string
object, this could be considered to be a bug in the *caller*. The caller
can fix this problem by dup'ing the string at the point when they pass
it in, or dup'ing it later before changing it.

Again, this is not a hard-and-fast rule. Sometimes defensive dup'ing is
reasonable. For example, Ruby's Hash object has a special-case for
string keys: if you pass in an unfrozen string as a hash key, then the
string is dup'd and frozen before being used as the key.

This may (a) lead to surprising behaviour, and (b) doesn't solve the
general problem (*). However strings are very commonly used as hash
keys, and they are usually short, so it seems a reasonable thing to do.

Regards,

Brian.

(*) In the case of a hash with string keys, if you mutated one of those
keys then it could end up being on the wrong hash chain, and become
impossible to retrieve it from the hash using hash["key"]

So it's a question of which of two behaviours is least undesirable:
objects disappearing from the hash because you forgot to call #rehash,
or hashes working "right" with string keys but "wrong" with other keys.

irb(main):001:0> k = [1]
=> [1]
irb(main):002:0> h = {k => 1, [2] => 2}
=> {[1]=>1, [2]=>2}
irb(main):003:0> h[[1]]
=> 1
irb(main):004:0> k << 2
=> [1, 2]
irb(main):005:0> h
=> {[1, 2]=>1, [2]=>2}
irb(main):006:0> h[[1,2]]
=> nil <<< WHOOPS!
irb(main):007:0> h.rehash
=> {[1, 2]=>1, [2]=>2}
irb(main):008:0> h[[1,2]]
=> 1

···

--
Posted via http://www.ruby-forum.com/\.

23

Not sure what you mean because the code makes sure that the argument
is *not* mutated.

Or did you mean "caller" when you wrote "I" above? There are
different policies to handle that as this thread has shown. Generally
I tend to follow the line that the caller is the owner of the object
unless otherwise stated. This also means that he is free to change it
at any time and consequently it is the responsibility of the receiver
to take whatever measures if he needs to make sure he keeps track of
the unmodified state.

IMHO this is a reasonable policy because if ownership passing would be
the default you would have to copy always as the caller if you need to
continue to work with the object - even if receivers just use the
object temporary and do not store references for longer. This would be
a waste of resources.

In practice this is usually not an issue for me so IMHO the discussion
has a strong "theoretical" aspect. OTOH it's good to reason about
these things from time to time. Helps clarify some things and avoid
mistakes.

Cheers

robert

···

2009/1/22 Stefan Rusterholz <apeiros@gmx.net>:

Robert Klemme wrote:

class MyClass
   def initialize(name)
     @name = name.dupf
   end
end

I'd vote against this. It looks to me like a great way to confuse users
and complicate interfaces. I'd rather go towards transparency.
Generally I'd not mutate arguments, only the receiver. If there's a
valid case to mutate an argument, it should be documented and evident.
The user then has to provide a duplicate if he still needs the original.

--
remember.guy do |as, often| as.you_can - without end

24

(Ooops, sorry if this message makes it through in duplicate with a
"strange" from, but I mixed up my "identities" in my newsreader...)

Tom Cloyd wrote:

If this is an utterly dumb question, just ignore it. However, I AM
perplexed by this response. Here's why:

i agree with you.

I have objects that get a string, process/clean it for printing/display.
(That is the whole purpose of centralizing data and behaviour into
classes.)

You could also design some "clever" accessors, like this :

class Class
  def attr_duplicator(*members)
    members.each do |m|
      self.class_eval(<<-EOM)
        def #{m}
          @#{m}.dup
        end
        def #{m}=(v)
          @#{m} = v.dup
        end
      EOM
    end
  end
  nil
end

(Could even bail if the writer is passed something else than a string.)

A long winded example :

class Blah ; attr_duplicator :blix ; end

=> nil

t = Blah.new()

=> #<Blah:0x000000016fce98>

z = "abc"

=> "abc"

z.__id__

=> 12031220

t.blix = z

=> "abc"

t.instance_variable_get("@blix").__id__

=> 12015560

t.blix.__id__

=> 11609640

t.blix.__id__

=> 11604360

t.blix << "aaa"

=> "abcaaa"

t.blix

=> "abc"

a = t.blix

=> "abc"

a << "aaa"

=> "abcaaa"

t.blix

=> "abc"

As usual, not sure about the performance impact, though, especially if
you manipulate very big string objects...

Fred

···

Le Thu, 22 Jan 2009 00:05:58 -0500, RK Sentinel a écrit :
--
So I open my door to my enemies
And I ask could we wipe the slate clean
But they tell me to please go fuck myself
You know you just can't win (Pink Floyd, Lost For Words)

25

I would not call this "incompetence": we live and learn. Basically this is a typical trade off issue: you trade efficiency (no copy) for safety (no aliasing). As often with trade offs there is no clear 100% rule which exactly tells you what is *always* correct. Instead you have to think about it - when creating something like a library even more so - and then deliberately decide which way you go.

Kind regards

  robert

···

On 22.01.2009 15:11, RK Sentinel wrote:

As and when i discover such bugs in my code, I start adding dup(), and yes sometimes these lines bomb when another datatype is passed (I had asked this in a thread recently: respond_to? dup was passing, but the dup was failing).

Anyway, i realize its more my incompetence, and i must be careful with destructive methods, but I just thought maybe there's some other way to do this, so i am not leaving it to my memory.

--
remember.guy do |as, often| as.you_can - without end

26

RK Sentinel wrote:

Thanks for all the helpful replies. Its my first venture: a widget
library.

Here's an example: Sometimes a string is passed to a class, say, using
its set_buffer method (which is an _optional_ method).

set_buffer just assigns it to @buffer. But deep within the class this
variable is being edited using insert() or slice!() (and this IS
necessary) since the widget is an editing widget.

Thanks, so I guess the API is something like this:

  edit_field.buffer = "foo"

  ... some time later, after user has clicked OK ...

  f.write(edit_field.buffer)

Now, if there is a compelling reason for this object to perform
"in-place" editing on the buffer then by all means do, and document
this, but it will lead to the aliasing problems you describe.

It may be simpler and safer just to use non-destructive methods inside
your class.

  # destructive
  @buffer.slice!(x,y)
  # non-destructive alternative
  @buffer = @buffer.slice(x,y)

  # destructive
  @buffer.insert(pos, text)
  # non-destructive alternative
  @buffer = buffer[0,pos] + text + buffer[pos..-1]

In effect, this is doing a 'dup' each time. It has to; since Ruby
doesn't do reference-counting it has no idea whether any other object in
the system is holding a reference to the original object or not.

The only problem with this is if @buffer is a multi-megabyte object and
you don't want to keep copying it. In this case, doing a single dup
up-front would allow you to use the destructive methods safely.

class Editor
  def buffer
    @buffer
  end
  def buffer=(x)
    @buffer = x.dup
  end
end

The overhead of a single copy is small, and in any case this is probably
what is needed here (e.g. if the user makes some edits but clicks
'cancel' instead of 'save' then you may want to keep the old string
untouched)

You could try deferring the dup until the first time you call a
destructive method on the string, but the complexity overhead is
unlikely to be worth it.

···

--
Posted via http://www.ruby-forum.com/\.

27

Robert Dober wrote:

Basically yes.
But one has to be careful, as we somehow have the instinct not to
create lots of short time objects.

To Tom, sorry for the sloppy abbreaviation, but it means incremental
Garbage Collector, applying different strategies of collection
depending on object age.
If you have 45m to spare, there was a most interesting talk at
Rubytalk by Glenn Vanderbourg, have a look by all means:
http://rubyconf2008.confreaks.com/how-ruby-can-be-fast.html

Cheers
Robert

Robert,

Thanks! I felt bad about the question, because reading on in to a couple of the posts following, I figured it out, and realized I DID know what the abbreviation meant. The "incremental" threw me off a bit. Thanks for the link. I'd love to go check out that link, and will likely do so later today.

Thank again for your help, as always. I'm endlessly grateful for the helpfulness of the folk on this list.

t.

···

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tom Cloyd, MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< tc@tomcloyd.com >> (email)
<< TomCloyd.com >> (website) << sleightmind.wordpress.com >> (mental health weblog)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

28

* Sebastian Hungerecker <sepp2k@googlemail.com> (00:27) schrieb:

Symbol#to_proc is new in ruby core since 1.8.7+. It had been previously
defined by ActiveSupport and, I think, facets.

WTF? What's this __[gs]etobj__ about?

Those are methods defined by DelegateClass(). [...]

Thanks, I never saw DelegateClass before, now I understand.

mfg, simon .... l

29

Beware: this may not solve your problem. It will work if the passed-in
object is itself a String, but not if it's some other object which
contains Strings.

I guess I'll just have to remember not to modify a string I take from
another class. I don't feel to happy or reassured about that since it's
been already causing me some grief.

Maybe I am just used to a bad practice coming from another language
whereas for others it's second nature.

···

--
Posted via http://www.ruby-forum.com/\.

30

Brian Candler wrote:

The overhead of a single copy is small, and in any case this is probably
what is needed here (e.g. if the user makes some edits but clicks
'cancel' instead of 'save' then you may want to keep the old string
untouched)

You could try deferring the dup until the first time you call a
destructive method on the string, but the complexity overhead is
unlikely to be worth it.

Yes, I've got the set_buffer doing a dup (if its a string).

At the same time, the get_buffer also does a dup, since often the Field
is created blank (i did mention that set_buffer is an optional method
for editing a default value, if present).

Its a real TextField or Field. So you would be typing away in the field.
Each character you type is inserted in (or removed if its del or BS) -
exactly as I am typing away in this editbox.

The CopyOnWriteString a impressive, shows what all can be done with
Ruby.

···

--
Posted via http://www.ruby-forum.com/\.

31

Brian Candler wrote:

You could try deferring the dup until the first time you call a
destructive method on the string, but the complexity overhead is
unlikely to be worth it.

Careful. Intuition is worse than useless here. The only way to know is
to measure the particular case in question.

class Person
  def initialize(name)
    @name = name
  end
  def name_cow
    CopyOnWriteString.new(@name)
  end
  def name_dup
    @name.dup
  end
end

require 'benchmark'

n = 10_000
sizes = [100, 1000, 10_000, 100_000]
objects = sizes.inject(Hash.new) { |acc, size|
  acc.merge!(size => Person.new("x"*size))
}

sizes.each { |size|
  object = objects[size]
  puts "-"*40
  puts "iterations: #{n} size: #{size}"
  Benchmark.bm { |x|
    x.report("cow w/o change") {
      n.times { object.name_cow }
    }
    x.report("dup w/o change") {
      n.times { object.name_dup }
    }
    x.report("cow w/ change") {
      n.times { object.name_cow << "y" }
    }
    x.report("dup w/ change") {
      n.times { object.name_dup << "y" }
    }
  }
}

ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-cygwin]

···

----------------------------------------
iterations: 10000 size: 100
      user system total real
cow w/o change 0.031000 0.000000 0.031000 ( 0.031000)
dup w/o change 0.032000 0.000000 0.032000 ( 0.031000)
cow w/ change 0.171000 0.000000 0.171000 ( 0.172000)
dup w/ change 0.047000 0.000000 0.047000 ( 0.047000)
----------------------------------------
iterations: 10000 size: 1000
      user system total real
cow w/o change 0.032000 0.000000 0.032000 ( 0.031000)
dup w/o change 0.046000 0.000000 0.046000 ( 0.047000)
cow w/ change 0.172000 0.000000 0.172000 ( 0.172000)
dup w/ change 0.063000 0.000000 0.063000 ( 0.062000)
----------------------------------------
iterations: 10000 size: 10000
      user system total real
cow w/o change 0.031000 0.000000 0.031000 ( 0.032000)
dup w/o change 0.109000 0.000000 0.109000 ( 0.109000)
cow w/ change 0.282000 0.000000 0.282000 ( 0.281000)
dup w/ change 0.156000 0.000000 0.156000 ( 0.156000)
----------------------------------------
iterations: 10000 size: 100000
      user system total real
cow w/o change 0.031000 0.000000 0.031000 ( 0.032000)
dup w/o change 0.672000 0.000000 0.672000 ( 0.672000)
cow w/ change 1.406000 0.000000 1.406000 ( 1.406000)
dup w/ change 1.219000 0.000000 1.219000 ( 1.219000)

Destructive methods are less common in real code, and especially so when
the string comes from a attr_reader method. It is likely that the case
to optimize is the non-destructive call (the first of each quadruplet
above). But we have to profile the specific situation.
--
Posted via http://www.ruby-forum.com/\.

32

RK Sentinel wrote:

I guess I'll just have to remember not to modify a string I take from
another class.

Or as was said earlier on: simply don't use destructive string methods
unless you really have to. Pretend that all strings are frozen.

You can enforce this easily enough in your unit tests: e.g. you could
pass in strings that really are frozen, and check that your code still
works :slight_smile:

···

--
Posted via http://www.ruby-forum.com/\.

33

Brian Candler wrote:

RK Sentinel wrote:

I guess I'll just have to remember not to modify a string I take from
another class.

Or as was said earlier on: simply don't use destructive string methods
unless you really have to. Pretend that all strings are frozen.

Yes, I did mention too somewhere to use gsub! etc with caution. Fine in
local variables, strings that are not exposed.

He says he modifies some object ...and to his great surprise the object
indeed gets modified.

:slight_smile: Nice way of putting it. But this happened across objects, and I was
wondering whether there was some other solution to the problem which did
not require my having to remember not to modify a string taken from
elsewhere.

···

--
Posted via http://www.ruby-forum.com/\.

34

Well, this is only half of the story: it does not save you from outside code changing the instance under your hands. Aliasing works both ways, i.e. you can screw up the receiver but also the caller. :slight_smile:

Cheers

  robert

···

On 22.01.2009 12:23, Brian Candler wrote:

RK Sentinel wrote:

I guess I'll just have to remember not to modify a string I take from another class.

Or as was said earlier on: simply don't use destructive string methods unless you really have to. Pretend that all strings are frozen.

You can enforce this easily enough in your unit tests: e.g. you could pass in strings that really are frozen, and check that your code still works :slight_smile:

--
remember.guy do |as, often| as.you_can - without end