Rb_str_new with custom subclass of rb_cString

Samuel_Williams · 18 March 2017 03:12

Is it possible to construct a string with a custom subclass or replace the
klass pointer of a VALUE in an existing one?

I see rb_str_new_with_class but it actually takes an object and uses its
class rather than a class directly which might work but is a bit of a hack.
Ideally would be able to change the class of the object... but perhaps
that's asking too much?

Thanks,
Samuel

Eric_Wong · 18 March 2017 08:41

Is it possible to construct a string with a custom subclass or replace the
klass pointer of a VALUE in an existing one?

You can always use:

rb_funcall(your_subclass, ... rb_intern("new"))

I see rb_str_new_with_class but it actually takes an object and uses its
class rather than a class directly which might work but is a bit of a hack.

I guess it works, but yeah, hacky...

Ideally would be able to change the class of the object... but perhaps
that's asking too much?

That might make future (and maybe today's) optimizations more
difficult.

The only place where an object's class is malleable is IO,
because that maps to the OS behavior (at least on *nix).

Keep in mind subclassing some highly-used core classes (such as
String) is a bad idea, performance-wise. The VM special-cases
for some core classes, but those optimizations will be disabled
in your subclass. You can see most of these VM optimizations in
insns.def in the C Ruby sources.

···

Samuel Williams <space.ship.traveller@gmail.com> wrote:

Dmitriy_Non · 19 March 2017 05:17

Hello!
Excuse me, may be I don't quite understand you.
Why do you want use native code to create an String subclass instance?

Doesn't this work for you?

class MyStr < String; end
MyStr.new('Hello, World!')

⁣

На 18 марта 2017 г., 6:13, в 6:13, Samuel Williams <space.ship.traveller@gmail.com> написал:п>Is it possible to construct a string with a custom subclass or replace

···

the
klass pointer of a VALUE in an existing one?

I see rb_str_new_with_class but it actually takes an object and uses
its
class rather than a class directly which might work but is a bit of a
hack.
Ideally would be able to change the class of the object... but perhaps
that's asking too much?

Thanks,
Samuel

------------------------------------------------------------------------

Unsubscribe:
<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Samuel_Williams · 19 March 2017 21:27

Doing

class MyStr < String; end
MyStr.new('Hello, World!')

makes this hot code path too inefficient.

I found I can use rb_obj_reveal to change class, it worked perfectly
and performance was maintained. Not sure if this is a good idea but
according to the docs its okay, but the source code says not to use
it.

Thanks

···

On 19 March 2017 at 18:17, Dmitriy Non <non.dmitriy@gmail.com> wrote:

Hello!
Excuse me, may be I don't quite understand you.
Why do you want use native code to create an String subclass instance?

Doesn't this work for you?
class MyStr < String; end
MyStr.new('Hello, World!')
На 18 марта 2017 г., 6:13, в 6:13, Samuel Williams
<space.ship.traveller@gmail.com> написал:п>Is it possible to construct a
string with a custom subclass or replace

the
klass pointer of a VALUE in an existing one?

I see rb_str_new_with_class but it actually takes an object and uses
its
class rather than a class directly which might work but is a bit of a
hack.
Ideally would be able to change the class of the object... but perhaps
that's asking too much?

Thanks,
Samuel

------------------------------------------------------------------------

Unsubscribe:
<mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Eric_Wong · 20 March 2017 00:27

rb_obj_reveal is intended for hidden objects (klass == 0);
I don't think ruby-core can guarantee it will remain working
for changing non-hidden objects.

Anyways, why do you need to subclass String? You seem to care
about performance, and you may lose a lot of it when subclassing.

···

Samuel Williams <space.ship.traveller@gmail.com> wrote:

Doing
class MyStr < String; end
MyStr.new('Hello, World!')
makes this hot code path too inefficient.

I found I can use rb_obj_reveal to change class, it worked perfectly
and performance was maintained. Not sure if this is a good idea but
according to the docs its okay, but the source code says not to use
it.

Samuel_Williams · 20 March 2017 04:27

Anyways, why do you need to subclass String? You seem to care
about performance, and you may lose a lot of it when subclassing.

I've got quite a few benchmarks that show that significant performance
is gained by my code. But, it's still a work in progress. I don't
assert that it's good or the right way, but I just see how I can
improve real world performance for our use case.

I'm tracking whether strings need to be escaped on output (e.g.
something similar to CGI.escape_html). It's almost a zero-cost
operation during parsing of the source markup, so I can avoid it
entirely. It's a big performance win, page render time improve by 50%.
The method needs to be low-cost, instantiating a custom sub-class of
string blows away any performance gains.

You can see a bit more how it works here:

So, when parsing Markup (approximately HTML), we track if any entities
were seen:

github.com

ioquatix/trenni/blob/342e7fe50bf35a937437e1ce706d72af808ac2b6/ext/trenni/markup.rl#L72


      
          		has_entities = 1;
          		
          		codepoint = strtoul(entity.begin, (char **)&entity.end, 16);
          		
          		Trenni_append_codepoint(&pcdata, encoding, codepoint);
          	}
          	
          	action entity_number {
          		entity.end = p;
          		
          		has_entities = 1;
          		
          		codepoint = strtoul(entity.begin, (char **)&entity.end, 10);
          		
          		Trenni_append_codepoint(&pcdata, encoding, codepoint);
          	}
          	
          	action doctype_begin {
          		doctype.begin = p;
          	}

If entities are not seen, we don't need to escape the string on
output, we mark it with Trenni_markup_safe:

github.com

ioquatix/trenni/blob/342e7fe50bf35a937437e1ce706d72af808ac2b6/ext/trenni/markup.rl#L155


      
          	action attribute_value {
          		has_value = 1;
          	}
          	
          	action attribute_empty {
          		has_value = 2;
          	}
          	
          	action attribute {
          		if (has_value == 1) {
          			rb_funcall(delegate, id_attribute, 2, Trenni_token(identifier, encoding), Trenni_markup_safe(pcdata, has_entities));
          		} else if (has_value == 2) {
          			rb_funcall(delegate, id_attribute, 2, Trenni_token(identifier, encoding), empty_string);
          		} else {
          			rb_funcall(delegate, id_attribute, 2, Trenni_token(identifier, encoding), Qtrue);
          		}
          	}
          	
          	action tag_opening_end {
          		rb_funcall(delegate, id_open_tag_end, 1, self_closing == 1 ? Qtrue : Qfalse);
          	}

github.com

ioquatix/trenni/blob/342e7fe50bf35a937437e1ce706d72af808ac2b6/ext/trenni/trenni.h#L71-L78


      
          static inline VALUE Trenni_markup_safe(VALUE string, unsigned has_entities) {
          	if (!has_entities) {
          		// Apparently should not use this to change klass, but it's exactly what we need here to make things lightning fast.
          		rb_obj_reveal(string, rb_Trenni_MarkupString);
          	}
          	
          	return string;
          }

On output, we use this information, e.g. when generating tags:

github.com

ioquatix/trenni/blob/342e7fe50bf35a937437e1ce706d72af808ac2b6/ext/trenni/tag.c#L52-L53


      
          			if (rb_obj_is_kind_of(value, rb_Trenni_Markup)) {
          				rb_str_append(buffer, value);

Additionally, I found that CGI.escape_html is a bit slow, can be about
20% faster. I haven't finished my implementation since I'm just
exploring the various available optimisations. But, the tangible
result is page render times for a complex page on my laptop went from
about 15ms down to 6-7ms with native C code and avoiding escape_html
where possible.

Thanks for your interest.

···

On 20 March 2017 at 13:27, Eric Wong <e@80x24.org> wrote:

Samuel Williams <space.ship.traveller@gmail.com> wrote:
Doing
class MyStr < String; end
MyStr.new('Hello, World!')
makes this hot code path too inefficient.

I found I can use rb_obj_reveal to change class, it worked perfectly
and performance was maintained. Not sure if this is a good idea but
according to the docs its okay, but the source code says not to use
it.
rb_obj_reveal is intended for hidden objects (klass == 0);
I don't think ruby-core can guarantee it will remain working
for changing non-hidden objects.

Anyways, why do you need to subclass String? You seem to care
about performance, and you may lose a lot of it when subclassing.

Unsubscribe: <mailto:ruby-talk-request@ruby-lang.org?subject=unsubscribe>
<http://lists.ruby-lang.org/cgi-bin/mailman/options/ruby-talk>

Topic		Replies	Views
How to create classes automatically ruby-talk	3	162	3 August 2006
Creating Ruby objects from C ruby-talk	2	128	11 March 2007
Idea: klass.from_s(str) ruby-talk	17	178	25 August 2005
[RCR] subclasses of string as hash keys ruby-talk	14	106	17 May 2004
Redefining rb_str_init() ruby-talk	3	90	4 December 2006

Rb_str_new with custom subclass of rb_cString

Related topics