# Unicode escaping fun & games

**URL:** <https://rubytalk.org/t/unicode-escaping-fun-games/53015>\
**Category:** ruby-talk\
**Created:** [23 April 2009 16:19 UTC](https://rubytalk.org/t/unicode-escaping-fun-games/53015 "2009-04-23T16:19:17Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Andrew\_S\_Townley1](https://avatars.discourse-cdn.com/v4/letter/a/b4bc9f/32.png) [@Andrew\_S\_Townley1](https://rubytalk.org/u/Andrew_S_Townley1)\
**Post date:** [23 April 2009 16:19 UTC](https://rubytalk.org/t/unicode-escaping-fun-games/53015/1 "2009-04-23T16:19:17Z")

</div>

Hi folks,

After my last question, I finally sat down and figured out how to easily  
do the kinds of conversions I wanted (at least the Unicode UTF-8 part).  
Here's what I came up with in the event that it may be useful to others  
having to exchange encoded 7-bit data across environments.

excalibur$ cat utf8.rb  
# Created: Thu Apr 23 17:03:23 IST 2009

> **···**
>
> #  
> # This is some quick code to deal with UTF-8 manipulation and  
> # serialization of 7-bit ASCII representations.
> 
> $KCODE='u'  
> require 'jcode'
> 
> def utf8\_escape(str)  
> &nbsp;&nbsp;s = ""  
> &nbsp;&nbsp;str.each\_char do |c|  
> &nbsp;&nbsp;&nbsp;&nbsp;x = c.unpack("C")[0]  
> &nbsp;&nbsp;&nbsp;&nbsp;if x \< 128  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;s \<\< c  
> &nbsp;&nbsp;&nbsp;&nbsp;else  
> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;s \<\< "\\u%04x" % c.unpack("U")[0]  
> &nbsp;&nbsp;&nbsp;&nbsp;end  
> &nbsp;&nbsp;end  
> &nbsp;&nbsp;s  
> end
> 
> def utf8\_unpack(str)  
> &nbsp;&nbsp;str.gsub(/\\u([0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F])/) do  
> &nbsp;&nbsp;&nbsp;&nbsp;[$1.hex].pack("U\*")  
> &nbsp;&nbsp;end  
> end
> 
> Running it:
> 
> excalibur$ irb  
> irb(main):001:0\> require 'utf8'  
> =\> true  
> irb(main):002:0\> s = "Hello €!"  
> =\> "Hello €!"  
> irb(main):003:0\> t = utf8\_escape(s)  
> =\> "Hello \\u20ac!"  
> irb(main):004:0\> u = utf8\_unpack(t)  
> =\> "Hello €!"  
> irb(main):005:0\> s == u  
> =\> true
> 
> excalibur$ irb  
> irb(main):001:0\> s = "àcA绋féà"  
> =\> "\303\240cA\347\273\213f\303\251\303\240"  
> irb(main):002:0\> require 'utf8'  
> =\> true  
> irb(main):003:0\> s = "àcA绋féà"  
> =\> "àcA绋féà"  
> irb(main):004:0\> t = utf8\_escape(s)  
> =\> "\\u00e0cA\\u7ecbf\\u00e9\\u00e0"  
> irb(main):005:0\> u = utf8\_unpack(t)  
> =\> "àcA绋féà"  
> irb(main):006:0\> s == u  
> =\> true
> 
> It may not be 100% bullet-proof, but it works for some simple examples  
> that I could find, so this may be as far as I need to go with that part.  
> The next step is to roll this into a one-pass string escaping routine so  
> you don't need to do a bunch of gsub calls.
> 
> Any suggestions, comments and improvements are welcome.
> 
> Cheers,
> 
> ast  
> --  
> Andrew S. Townley \<[ast@atownley.org](mailto:ast@atownley.org)\>  
> [http://atownley.org](http://atownley.org)