# Character encoding in 1.9

**URL:** <https://rubytalk.org/t/character-encoding-in-1-9/66937>\
**Category:** ruby-talk\
**Created:** [24 December 2012 20:34 UTC](https://rubytalk.org/t/character-encoding-in-1-9/66937 "2012-12-24T20:34:36Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![7stud2](https://avatars.discourse-cdn.com/v4/letter/7/9de053/32.png) [@7stud2](https://rubytalk.org/u/7stud2)\
**Post date:** [24 December 2012 20:34 UTC](https://rubytalk.org/t/character-encoding-in-1-9/66937/1 "2012-12-24T20:34:36Z")

</div>

I have some code in 1.8 that would strip certain special characters out  
of a string (escape sequences in Telnet specifically, such as  
\377\373\001).

I've moved to 1.9 and now that code fails with "invalid multibyte  
escape".

I've looked at some of the tutorials regarding encodings in 1.9 and to  
be honest I find them very intimidating. I'm hoping someone can suggest  
a quick fix?

Here's the code:

[3,4,5,6,7].each do |n|  
&nbsp;&nbsp;&nbsp;&nbsp;line.gsub!(/#{('\377\37' + n.to\_s + '\001')}/,"")  
end

Many thanks in advance!

- Andrew

> **···**
>
> --  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

---

<div class="post-metadata">

**Author:** ![Nathan\_Beyer](https://avatars.discourse-cdn.com/v4/letter/n/13edae/32.png) [@Nathan\_Beyer](https://rubytalk.org/u/Nathan_Beyer)\
**Post date:** [25 December 2012 04:28 UTC](https://rubytalk.org/t/character-encoding-in-1-9/66937/2 "2012-12-25T04:28:43Z")

</div>

Check out the Regexp documentation, specifically the 'new'/'compile'  
method: [Class: Regexp (Ruby 1.9.3)](http://www.ruby-doc.org/core-1.9.3/Regexp.html#method-c-new%5C). I  
think you'll need to use Regexp.compile to create an "encoding-less"  
regular expression.

I would guess something like this -

[3,4,5,6,7].each do |n|  
&nbsp;&nbsp;&nbsp;&nbsp;line.gsub!(Regexp.compile('\377\37' + n.to\_s + '\001', nil, 'n'),"")  
end

You'll have to fiddle with it, as I didn't have a chance to test it out to  
see if this actually works. I'd have to know the encoding the 'line'  
variable to accurately test it anyway.

> **···**
>
> On Mon, Dec 24, 2012 at 2:34 PM, Andrew S. \<lists@ruby-forum.com\> wrote:
> 
> > I have some code in 1.8 that would strip certain special characters out  
> > of a string (escape sequences in Telnet specifically, such as  
> > \377\373\001).
> > 
> > I've moved to 1.9 and now that code fails with "invalid multibyte  
> > escape".
> > 
> > I've looked at some of the tutorials regarding encodings in 1.9 and to  
> > be honest I find them very intimidating. I'm hoping someone can suggest  
> > a quick fix?
> > 
> > Here's the code:
> > 
> > [3,4,5,6,7].each do |n|  
> > &nbsp;&nbsp;&nbsp;&nbsp;line.gsub!(/#{('\377\37' + n.to\_s + '\001')}/,"")  
> > end
> > 
> > Many thanks in advance!
> > 
> > - Andrew
> > 
> > --  
> > Posted via [http://www.ruby-forum.com/\](http://www.ruby-forum.com/%5C).

---

<div class="post-metadata">

**Author:** ![7stud2](https://avatars.discourse-cdn.com/v4/letter/7/9de053/32.png) [@7stud2](https://rubytalk.org/u/7stud2)\
**Post date:** [25 December 2012 08:15 UTC](https://rubytalk.org/t/character-encoding-in-1-9/66937/3 "2012-12-25T08:15:46Z")

</div>

line = "hello\377\373\001 \377\374\001 \377\375\001world"

(3..7).each do |n|  
&nbsp;&nbsp;pattern = "\377" \<\< (0370 + n).chr \<\< "\001"  
&nbsp;&nbsp;line.gsub! pattern, " "  
&nbsp;&nbsp;puts line  
end

--output:--  
hello ?? ??world  
hello ??world  
hello world  
hello world  
hello world

> **···**
>
> --  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

---

<div class="post-metadata">

**Author:** ![7stud2](https://avatars.discourse-cdn.com/v4/letter/7/9de053/32.png) [@7stud2](https://rubytalk.org/u/7stud2)\
**Post date:** [25 December 2012 09:09 UTC](https://rubytalk.org/t/character-encoding-in-1-9/66937/4 "2012-12-25T09:09:08Z")

</div>

line = "hello\377\373\001 \377\374\001 \377\375\001world"  
p line

regex = /\377[\373-\377]\001/

line.gsub! regex, ''  
p line

--output:--  
"hello\xFF\xFB\x01 \xFF\xFC\x01 \xFF\xFD\x01world"  
"hello world"

> **···**
>
> --  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).