String doesnt auto dup on modification

ruby-talk
1

I'm writing my first largeish app. One issue that gets me frequently is
this:

I define a string in one class. Some other class references it, and
modifies it. I (somehow) expected that when another referer modifies the
reference, ruby would automatically dup() the string.

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

Is there some clean way of handling this that I am missing.

···

--
Posted via http://www.ruby-forum.com/.

2

- Do you habitually dup() your string ?

Is there some clean way of handling this that I am missing.

To continue:

In some critical places in my app I had done:

def set_value str
  @buffer = str.dup
end

def get_value
  @buffer.dup
end

Is this the norm? Do you do this generally, to avoid accidental changes
?

···

--
Posted via http://www.ruby-forum.com/\.

3

This is a well known "problem" with all languages that
have mutable strings. The solution is simple:

* Use destructive string methods only after profiling has shown
  that string manipulation is the bottleneck.

* Don't mutate a string after passing it across encapsulation
  boundaries.

Freezing certain strings can be beneficial in the same way
assertions are, habitually duping strings is a bad practice, IMO.

Stefan

···

2009/1/21 RK Sentinel <sentinel.2001@gmx.com>:

I'm writing my first largeish app. One issue that gets me frequently is
this:

I define a string in one class. Some other class references it, and
modifies it. I (somehow) expected that when another referer modifies the
reference, ruby would automatically dup() the string.

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

Is there some clean way of handling this that I am missing.

4

RK Sentinel wrote:

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

Generally, no.

Of course there is no contract to enforce this, but in many cases it
would be considered bad manners to modify an object which is passed in
as an argument.

If you only read the object, then it doesn't matter. If you need a
modified version, create a new object. Usually this doesn't require
'dup'.

  def foo(a_string)
    a_string << "/foo" # bad
    a_string = "#{a_string}/foo" # good
    a_string = a_string + "/foo" # good
  end

  DEFAULT_OPT = {:foo => "bar"}

  def bar(opt = {})
    opt[:foo] ||= "bar" # bad
    opt = DEFAULT_OPT.merge(opt) # good
  end

If you are paranoid, you can freeze DEFAULT_OPT and all its keys and
values.

Sometimes you will see frozen strings as an optimisation to reduce the
amount of garbage objects created:

  ...
  foo["bar"] # creates a new "bar" string every time round

  BAR = "bar".freeze
  ...
  foo[BAR] # always uses the same object

This probably won't make any noticeable difference except in the most
innermost of loops.

···

--
Posted via http://www.ruby-forum.com/\.

5

Stefan Lang wrote:

···

2009/1/21 RK Sentinel <sentinel.2001@gmx.com>:
> I define a string in one class. Some other class references it, and
> modifies it. I (somehow) expected that when another referer modifies the
> reference, ruby would automatically dup() the string.

This is a well known "problem" with all languages that
have mutable strings. The solution is simple:

If I understand the OP correctly, his problem has nothing to do with
strings.

He says he modifies some object ...and to his great surprise the object
indeed gets modified.
--
Posted via http://www.ruby-forum.com/\.

6

I try to, and I try to get rid of all references to the original
string as soon as possible.
This is because incremental GC works so well nowadays and allows for
some clean code.
Freezing a string seems like a good idea sometimes, but if that means
holding on to the object longer than needed this might not be such a
good idea after all.

R.

···

On Wed, Jan 21, 2009 at 8:15 PM, RK Sentinel <sentinel.2001@gmx.com> wrote:

I'm writing my first largeish app. One issue that gets me frequently is
this:

I define a string in one class. Some other class references it, and
modifies it. I (somehow) expected that when another referer modifies the
reference, ruby would automatically dup() the string.

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

--
It is change, continuing change, inevitable change, that is the
dominant factor in society today. No sensible decision can be made any
longer without taking into account not only the world as it is, but
the world as it will be ... ~ Isaac Asimov

7

RK Sentinel wrote:

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

One possibility is copy-on-write.

require 'delegate'

class CopyOnWriteString < DelegateClass(String)
  DESTRUCTIVE_METHODS =
    String.public_instance_methods(false).grep(/!/).map(&:to_sym) +
    [
      :=,
      :<<,
      :concat,
      :initialize_copy,
      :replace,
      :setbyte,
      # ... and probably others ...
    ]

  DESTRUCTIVE_METHODS.each { |m|
    define_method(m) { |*args, &block|
      __setobj__(__getobj__.dup)
      __getobj__.send(m, *args, &block)
    }
  }
end

class Person
  def initialize(name)
    @name = name
  end
  def name
    CopyOnWriteString.new(@name)
  end
end

person = Person.new("fred")
name = person.name

p name #=> "fred"
p person.name #=> "fred"

name << " flintstone"
p name #=> "fred flintstone"
p person.name #=> "fred"

(I've used some 1.8.7+ only features.)

···

--
Posted via http://www.ruby-forum.com/\.

8

Stefan Lang wrote:

  

I'm writing my first largeish app. One issue that gets me frequently is
this:

I define a string in one class. Some other class references it, and
modifies it. I (somehow) expected that when another referer modifies the
reference, ruby would automatically dup() the string.

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

Is there some clean way of handling this that I am missing.
    
This is a well known "problem" with all languages that
have mutable strings. The solution is simple:

* Use destructive string methods only after profiling has shown
  that string manipulation is the bottleneck.

* Don't mutate a string after passing it across encapsulation
  boundaries.

Freezing certain strings can be beneficial in the same way
assertions are, habitually duping strings is a bad practice, IMO.

Stefan

If this is an utterly dumb question, just ignore it. However, I AM perplexed by this response. Here's why:

I thought it was OK for an object to receive input, and output a modified version of same. If they don't get to do that, their use seems rather limited. In my current app, I create a log object, and various classes write to it. I don't create new objects every time I want to add a log entry. Why would I do that? Makes no sense to me. I might want to do exactly the same thing to a string. You seem to be saying this is bad form. I can see that there are cases where you want the string NOT to be modified, but you see to be saying that to modify the original string at all is bad.

It makes perfect sense to me to pass an object (string, in this case) across an encapsulation boundary specifically to modify it.

What am I missing here?

Thanks, if you can help me out!

Tom

···

2009/1/21 RK Sentinel <sentinel.2001@gmx.com>:

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tom Cloyd, MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< tc@tomcloyd.com >> (email)
<< TomCloyd.com >> (website) << sleightmind.wordpress.com >> (mental health weblog)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

9

RK Sentinel wrote:

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

Generally, no.

Same here.

Of course there is no contract to enforce this, but in many cases it would be considered bad manners to modify an object which is passed in as an argument.

Depends: for example, if you have a method that is supposed to dump something to a stream (IO and friends) which only uses << you can as well use String there.

If you only read the object, then it doesn't matter.

That may be true for methods but if you need to store a String as instance variable then I tend to dup it if the application is larger. You can even automate conditional dup'ing by doing something like this

class Object
   def dupf
     frozen? ? self : dup
   end
end

and then

class MyClass
   def initialize(name)
     @name = name.dupf
   end
end

Kind regards

  robert

···

On 21.01.2009 22:57, Brian Candler wrote:

--
remember.guy do |as, often| as.you_can - without end

10

Stefan Lang wrote:

> I define a string in one class. Some other class references it, and
> modifies it. I (somehow) expected that when another referer modifies the
> reference, ruby would automatically dup() the string.

This is a well known "problem" with all languages that
have mutable strings. The solution is simple:

If I understand the OP correctly, his problem has nothing to do with
strings.

Sorry, that's plain wrong. He explicitly mentions Strings in several
places. The object that is mutated *is* a String.

He says he modifies some object ...and to his great surprise the object
indeed gets modified.

:slight_smile:

Cheers

robert

···

2009/1/22 Albert Schlef <albertschlef@gmail.com>:

2009/1/21 RK Sentinel <sentinel.2001@gmx.com>:

--
remember.guy do |as, often| as.you_can - without end

11

Robert Dober wrote:

···

On Wed, Jan 21, 2009 at 8:15 PM, RK Sentinel <sentinel.2001@gmx.com> wrote:
  

I'm writing my first largeish app. One issue that gets me frequently is
this:

I define a string in one class. Some other class references it, and
modifies it. I (somehow) expected that when another referer modifies the
reference, ruby would automatically dup() the string.

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?
    

I try to, and I try to get rid of all references to the original
string as soon as possible.
This is because incremental GC works so well nowadays and allows for
some clean code.
Freezing a string seems like a good idea sometimes, but if that means
holding on to the object longer than needed this might not be such a
good idea after all.

R.
  

Robert, for those of us who are considerably more clueless, what is "incremental GC"?

Thanks,

t.

--

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tom Cloyd, MS MA, LMHC - Private practice Psychotherapist
Bellingham, Washington, U.S.A: (360) 920-1226
<< tc@tomcloyd.com >> (email)
<< TomCloyd.com >> (website) << sleightmind.wordpress.com >> (mental health weblog)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

12

- Do you habitually dup() your string ?

I try to, and I try to get rid of all references to the original
string as soon as possible.
This is because incremental GC works so well nowadays and allows for
some clean code.
Freezing a string seems like a good idea sometimes, but if that means
holding on to the object longer than needed this might not be such a
good idea after all.

R.

Interesting point. So freezing a string prevents collection as long as
there are referers (obvious), but duping it helps release the original
one, but you still have a new string in memory. So net you still are
taking the same memory.

Is there a writeup on Ruby GC collection, my knowledge of GC is java
based, and it is 5 years old (based on the Inside the VM book and
various other articles on sun.com). Is ruby's GC "generational" ? In
which iirc, an older object would have moved to an older generation and
be less likely to be collected.

Any links to ruby's GC would be appreciated.

···

--
Posted via http://www.ruby-forum.com/\.

13

* Mike Gold <mike.gold.4433@gmail.com> (2009-01-23) schrieb:

    String.public_instance_methods(false).grep(/!/).map(&:to_sym) +

Is that a new feature? make the to_sym method act as block. Which to_sym
method?

  DESTRUCTIVE_METHODS.each { |m|
    define_method(m) { |*args, &block|
      __setobj__(__getobj__.dup)
      __getobj__.send(m, *args, &block)

WTF? What's this __[gs]etobj__ about?

(I've used some 1.8.7+ only features.)

mfg, simon .... tia

14

Tom Cloyd wrote:

Stefan Lang wrote:

aware of freeze().

assertions are, habitually duping strings is a bad practice, IMO.

Stefan

If this is an utterly dumb question, just ignore it. However, I AM
perplexed by this response. Here's why:

i agree with you.

I have objects that get a string, process/clean it for printing/display.
(That is the whole purpose of centralizing data and behaviour into
classes.)

Remembering I must not modify it is a big mental overhead and results
in strange things that I spend a lot of time tracking, till I find out
--- oh no the string got mutated over there. Now I must start dup()'ing
it -- okay, where all should I dup it ?

To the previous poster - yes, one does not have to use dup. One can
create a new string by changing the method from say gsub! to just
gsub() and take the return value. I include such situations when i say
dup().

···

--
Posted via http://www.ruby-forum.com/\.

15

Tom Cloyd wrote:

I thought it was OK for an object to receive input, and output a
modified version of same.

Do you mean "return the same object reference, after the object has been
modified", or "return a new object, which is a modified copy of the
original"?

If they don't get to do that, their use seems
rather limited. In my current app, I create a log object, and various
classes write to it. I don't create new objects every time I want to add
a log entry. Why would I do that? Makes no sense to me.

I'd consider a logger object as a sort of stream. You're just telling
the logger to "do" something every time you send it a message; you're
not really telling it to change into a different sort of logger. (Of
course, if the logger is logging to an underlying string buffer, then
changing that buffer is a desired side effect of logging, but the logger
itself is still the same)

I might want to
do exactly the same thing to a string. You seem to be saying this is bad
form. I can see that there are cases where you want the string NOT to be
modified, but you see to be saying that to modify the original string at
all is bad.

No, I'm not saying this. Sometimes it's useful to modify the string
passed in:

  def cleanup!(str)
    str.strip!
    str.replace("default") if str.empty?
  end

However I'd say this is not the usual case. More likely I'd write

  def cleanup(str)
    str = str.strip
    str.empty? "default" : str
  end

It makes perfect sense to me to pass an object (string, in this case)
across an encapsulation boundary specifically to modify it.

Yes, in some cases it does, and it's up to you to agree the 'contract'
in your documentation that that's what you'll do. I'm not saying it's
forbidden.

But this seems to be contrary to your original question, where you were
saying you were defensively dup'ing strings, on both input and output,
to avoid cases where they get mutated later by the caller or some other
object invoked by the caller.

I'm saying to avoid this problem, the caller would not pass a string to
object X, and *then* mutate it (e.g. by passing it to another object Y
which mutates it). And in practice, I find this is not normally a
problem, because normally objects do not mutate strings which are passed
into them as arguments.

This is not a hard and fast rule. It's just how things work out for me
in practice. It depends on your coding style, and whether you're coding
for yourself or coding libraries to be used by other people too.

Regards,

Brian.

···

--
Posted via http://www.ruby-forum.com/\.

16

instance variable then I tend to dup it if the application is larger.

You can even automate conditional dup'ing by doing something like this

class Object
   def dupf
     frozen? ? self : dup
   end
end

and then

class MyClass
   def initialize(name)
     @name = name.dupf
   end
end

Kind regards

  robert

thanks, this looks very helpful.

In response to the prev post by Brian, yes its a library for use by
others.

···

--
Posted via http://www.ruby-forum.com/\.

17

Robert Klemme wrote:

class MyClass
   def initialize(name)
     @name = name.dupf
   end
end

I'd vote against this. It looks to me like a great way to confuse users
and complicate interfaces. I'd rather go towards transparency.
Generally I'd not mutate arguments, only the receiver. If there's a
valid case to mutate an argument, it should be documented and evident.
The user then has to provide a duplicate if he still needs the original.

Just my 0.02€

Regards
Stefan Rusterholz

···

--
Posted via http://www.ruby-forum.com/\.

18

Thanks for all the helpful replies. Its my first venture: a widget
library.

Here's an example: Sometimes a string is passed to a class, say, using
its set_buffer method (which is an _optional_ method).

set_buffer just assigns it to @buffer. But deep within the class this
variable is being edited using insert() or slice!() (and this IS
necessary) since the widget is an editing widget. So its not so clear
when i added set_buffer method that this would happen.

As and when i discover such bugs in my code, I start adding dup(), and
yes sometimes these lines bomb when another datatype is passed (I had
asked this in a thread recently: respond_to? dup was passing, but the
dup was failing).

Yesterday, I had written an Action class which can be used to create a
menuitem or a button. The string used in the Action constructor can be
like "&Delete".
Button and Menuitem remove the "&" using a slice!(). And i was wondering
why the second usage was failing to find the "&".

Anyway, i realize its more my incompetence, and i must be careful with
destructive methods, but I just thought maybe there's some other way to
do this, so i am not leaving it to my memory.

Thanks again for the helpful replies. The ruby community really rocks
when it comes helping others.

···

--
Posted via http://www.ruby-forum.com/.

19

There's nothing wrong with it if the purpose of the method
is to manipulate the string and it's documented clearly.

Every rule has exceptions :slight_smile:

Stefan

···

2009/1/21 Tom Cloyd <tomcloyd@comcast.net>:

Stefan Lang wrote:

2009/1/21 RK Sentinel <sentinel.2001@gmx.com>:

I'm writing my first largeish app. One issue that gets me frequently is
this:

I define a string in one class. Some other class references it, and
modifies it. I (somehow) expected that when another referer modifies the
reference, ruby would automatically dup() the string.

Anyway, through trial and error, I start dup()'ing strings myself. I am
aware of freeze().

But would like to know how others handle this generally in large apps.

- Do you keep freezing Strings you make in your classes to avoid
accidental change

- Do you habitually dup() your string ?

Is there some clean way of handling this that I am missing.

This is a well known "problem" with all languages that
have mutable strings. The solution is simple:

* Use destructive string methods only after profiling has shown
that string manipulation is the bottleneck.

* Don't mutate a string after passing it across encapsulation
boundaries.

Freezing certain strings can be beneficial in the same way
assertions are, habitually duping strings is a bad practice, IMO.

Stefan

If this is an utterly dumb question, just ignore it. However, I AM perplexed
by this response. Here's why:

I thought it was OK for an object to receive input, and output a modified
version of same. If they don't get to do that, their use seems rather
limited. In my current app, I create a log object, and various classes write
to it. I don't create new objects every time I want to add a log entry. Why
would I do that? Makes no sense to me. I might want to do exactly the same
thing to a string. You seem to be saying this is bad form. I can see that
there are cases where you want the string NOT to be modified, but you see to
be saying that to modify the original string at all is bad.

It makes perfect sense to me to pass an object (string, in this case) across
an encapsulation boundary specifically to modify it.

What am I missing here?

20

Basically yes.
But one has to be careful, as we somehow have the instinct not to
create lots of short time objects.

To Tom, sorry for the sloppy abbreaviation, but it means incremental
Garbage Collector, applying different strategies of collection
depending on object age.
If you have 45m to spare, there was a most interesting talk at
Rubytalk by Glenn Vanderbourg, have a look by all means:
http://rubyconf2008.confreaks.com/how-ruby-can-be-fast.html

Cheers
Robert

···

--
It is change, continuing change, inevitable change, that is the
dominant factor in society today. No sensible decision can be made any
longer without taking into account not only the world as it is, but
the world as it will be ... ~ Isaac Asimov