The encoding of __FILE__ is always the same as Encoding.default_external
even if there is a magic column. Sometimes it is necessary to convert
the string into another encoding. Here is some code to demonstrate the
issue:
#coding: utf-8
# put the script in a not-pure-ascii path to see the difference
path = File.expand_path File.dirname __FILE__
puts RUBY_VERSION + ' ' + RUBY_PLATFORM
#=> "1.9.1 i386-mingw32" is my ruby version
puts path.encoding
#=> "GB2312" on my OS
# usually this "string.encode to, from" works,
# but HERE the new string's content bytes seems unchanged
puts \
path.encode 'utf-8', Encoding.default_external
path.force_encoding Encoding.default_external
puts path.encode 'utf-8'
# changed at last
On Jan 17, 12:55 am, Lui Kore <usur...@gmail.com> wrote:
The encoding of __FILE__ is always the same as Encoding.default_external
even if there is a magic column. Sometimes it is necessary to convert
the string into another encoding. Here is some code to demonstrate the
issue:
#coding: utf-8
# put the script in a not-pure-ascii path to see the difference
path = File.expand_path File.dirname __FILE__
puts RUBY\_VERSION \+ ' ' \+ RUBY\_PLATFORM
\#=> "1\.9\.1 i386\-mingw32" is my ruby version
puts path\.encoding
\#=> "GB2312" on my OS
\# usually this "string\.encode to, from" works,
\# but HERE the new string's content bytes seems unchanged
puts \\
path\.encode 'utf\-8', Encoding\.default\_external
path\.force\_encoding Encoding\.default\_external
puts path\.encode 'utf\-8'
\# changed at last
I believe the point you are missing is that String#encode does not change the String but it returns a new String with the desired encoding. If you want inplace modification you need to use String#encode! which does just that.
The encoding of __FILE__ is always the same as Encoding.default_external
even if there is a magic column. Sometimes it is necessary to convert
the string into another encoding. Here is some code to demonstrate the
issue:
#coding: utf-8
# put the script in a not-pure-ascii path to see the difference
path = File.expand_path File.dirname __FILE__
puts RUBY_VERSION + ' ' + RUBY_PLATFORM
#=> "1.9.1 i386-mingw32" is my ruby version
puts path.encoding
#=> "GB2312" on my OS
# usually this "string.encode to, from" works,
# but HERE the new string's content bytes seems unchanged
puts \
path.encode 'utf-8', Encoding.default_external
path.force_encoding Encoding.default_external
puts path.encode 'utf-8'
# changed at last
def enc s
s.encode 'utf-8', Encoding.default_external
end
p1 = File.expand_path File.dirname __FILE__
p2 = p1.dup
p2.force_encoding p2.encoding # strange, but makes it different
I know String#encode doesn't change the original string, but the result
is encoded.
To understand the problem, you should try in a gbk/shift-jis environment
with some Chinese or Japanese path.
The point is:
For some path p1 and p2,
when p1 == p2 and p1.encoding == p2.encoding,
p1.encode('utf-8') == p2.encode('utf-8') is not always true.
To describe it in a "encode!" version:
For some path p1 and p2,
when p1 == p2 and p1.encoding == p2.encoding,
p1.encode!('utf-8')
p2.encode!('utf-8')
p1 == p2 is still not always true
Robert Klemme wrote:
···
On 01/17/2010 04:55 AM, Lui Kore wrote:
#=> "1.9.1 i386-mingw32" is my ruby version
# changed at last
I believe the point you are missing is that String#encode does not
change the String but it returns a new String with the desired encoding.
If you want inplace modification you need to use String#encode! which
does just that.
Apparently I misread your posting, sorry. Is UTF-8 capable of
representing those Japanese or Chinese characters? I believe I
remember Matz saying that UTF-8 is insufficient to properly represent
Japanese characters. If this is the case then I guess all bets are
off and you get undefined behavior. Although it might be desirable to
get the same garbage it may not be worthwhile to ensure this purely
for efficiency reasons.
Kind regards
robert
···
2010/1/18 Lui Kore <usurffx@gmail.com>:
I know String#encode doesn't change the original string, but the result
is encoded.
To understand the problem, you should try in a gbk/shift-jis environment
with some Chinese or Japanese path.
The point is:
For some path p1 and p2,
when p1 == p2 and p1.encoding == p2.encoding,
p1.encode('utf-8') == p2.encode('utf-8') is not always true.
To describe it in a "encode!" version:
For some path p1 and p2,
when p1 == p2 and p1.encoding == p2.encoding,
p1.encode!('utf-8')
p2.encode!('utf-8')
p1 == p2 is still not always true