[ruby talk] Localization woes

ruby-talk
1

People,

I’m trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

The idea behind it is simple.

For example, for Spanish, create a file named sp.yaml and fill it with
key value pairs:

hello: hola
here: aquí
Spanish: Español

Then when I need to translate a key into a value I do something like this:

@msg = l(:Spanish)

So everything seems to work okay if I put plain characters in sp.yaml.

Webrick, however, spazzes out if I put non-English characters in sp.yaml:

script/server
=> Booting WEBrick…
./script/…/config/…/lib/localization.rb:62:in iconv': "叩 aqu鱈 como" (Iconv::IllegalSequence) from ./script/../config/../lib/localization.rb:62:inload_localized_strings’
from ./script/…/config/…/lib/localization.rb:61:in each' from ./script/../config/../lib/localization.rb:61:inload_localized_strings’
from ./script/…/config/…/lib/localization.rb:60:in open' from ./script/../config/../lib/localization.rb:60:inload_localized_strings’
from ./script/…/config/…/lib/localization.rb:52:in each' from ./script/../config/../lib/localization.rb:52:inload_localized_strings’
from ./script/…/config/…/config/environment.rb:59
… 8 levels…
from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in
require__' from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:inrequire’
from /opt/local/lib/ruby/gems/1.8/gems/activesupport-1.2.3/lib/active_support/dependencies.rb:214:in
`require’
from script/server:3
zmac11:~/pt/TThier/Languages/ruby/j413 oracle$

Notice the msg:
./script/…/config/…/lib/localization.rb:62:in `iconv’: “叩 aqu鱈 como”
(Iconv::IllegalSequence)

It was caused by this line in my sp.yaml:
you_are_here_as: usted está aquí como

Obviously it had problems with
está
and
aquí

I used emacs to create sp.yaml and emacs asked me which character set
encoding I want to use.

I told it utf-8 and emacs saved sp.yaml just fine.

I pulled the actual characters from google and used my mouse to copy
them into emacs:

http://translate.google.com/translate_t

I checked the encoding my browser was using on the google page and it
told me it was using: UTF8
which I’ll assume is the same as the utf-8 I’m using in emacs.

Do any of you have some sample localization yaml files which you’d be
willing to share with me?

How about some general troubleshooting ideas?

Thanks,
-Dan

2

Dan Bikle escribió:

People,

I’m trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

The idea behind it is simple.

For example, for Spanish, create a file named sp.yaml and fill it with
key value pairs:

hello: hola
here: aquí
Spanish: Español

Then when I need to translate a key into a value I do something like this:

@msg = l(:Spanish)

So everything seems to work okay if I put plain characters in sp.yaml.

Webrick, however, spazzes out if I put non-English characters in sp.yaml:

script/server
=> Booting WEBrick…
./script/…/config/…/lib/localization.rb:62:in iconv': "叩 aqu鱈 como" (Iconv::IllegalSequence) from ./script/../config/../lib/localization.rb:62:in load_localized_strings’
from ./script/…/config/…/lib/localization.rb:61:in each' from ./script/../config/../lib/localization.rb:61:in load_localized_strings’
from ./script/…/config/…/lib/localization.rb:60:in open' from ./script/../config/../lib/localization.rb:60:in load_localized_strings’
from ./script/…/config/…/lib/localization.rb:52:in each' from ./script/../config/../lib/localization.rb:52:in load_localized_strings’
from ./script/…/config/…/config/environment.rb:59
… 8 levels…
from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in
require__' from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in require’
from /opt/local/lib/ruby/gems/1.8/gems/activesupport-1.2.3/lib/active_support/dependencies.rb:214:in
`require’
from script/server:3
zmac11:~/pt/TThier/Languages/ruby/j413 oracle$

Notice the msg:
./script/…/config/…/lib/localization.rb:62:in `iconv’: “叩 aqu鱈 como”
(Iconv::IllegalSequence)

It was caused by this line in my sp.yaml:
you_are_here_as: usted está aquí como

Obviously it had problems with
está
and
aquí

I used emacs to create sp.yaml and emacs asked me which character set
encoding I want to use.

I told it utf-8 and emacs saved sp.yaml just fine.

I pulled the actual characters from google and used my mouse to copy
them into emacs:

http://translate.google.com/translate_t

I checked the encoding my browser was using on the google page and it
told me it was using: UTF8
which I’ll assume is the same as the utf-8 I’m using in emacs.

Do any of you have some sample localization yaml files which you’d be
willing to share with me?

How about some general troubleshooting ideas?

Thanks,
-Dan

Rails mailing list
Rails-1W37MKcQCpIf0INCOvqR/iCwEArCW2h5@public.gmane.org
http://lists.rubyonrails.org/mailman/listinfo/rails

I think that yaml is not the best file format for storing translations,
as I ran with a similar problem when trying to store some translations
with HTML embedded…
Maybe the’d better look for another file format for that.

3

Dan Bikle wrote:

People,

I'm trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

You will get better, faster, help for Rails at

http://lists.rubyonrails.org/mailman/listinfo/rails

···

--

http://www.ruby-doc.org - Ruby Help & Documentation
Ruby Code & Style - Ruby Code & Style: Writers wanted
http://www.rubystuff.com - The Ruby Store for Ruby Stuff
http://www.jamesbritt.com - Playing with Better Toys
http://www.30secondrule.com - Building Better Tools

4

People,

I fixed the problem.
The method which loads the .yaml files which contain the translations
is located here:
lib/localization.rb

Here is some syntax from the method:

  def self.load_localized_strings
    # Load language files
    Dir[RAILS_ROOT + '/lang/*.yaml'].each do |filename|
      filename =~ /(([a-z]+_?)+)\.yaml$/
      hash = YAML::load(File.read(filename))
      file_charset = hash['file_charset'] || 'ascii'

I made a small change and my problems evaporated:

  def self.load_localized_strings
    # Load language files
    Dir[RAILS_ROOT + '/lang/*.yaml'].each do |filename|
      filename =~ /(([a-z]+_?)+)\.yaml$/
      hash = YAML::load(File.read(filename))
      file_charset = hash['file_charset'] || 'utf-8'

-Dan

···

On 12/5/05, Dan Bikle <dan.bikle@gmail.com> wrote:

People,

I'm trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

The idea behind it is simple.

For example, for Spanish, create a file named sp.yaml and fill it with
key value pairs:

hello: hola
here: aquí
Spanish: Español

Then when I need to translate a key into a value I do something like this:

@msg = l(:Spanish)

So everything seems to work okay if I put plain characters in sp.yaml.

Webrick, however, spazzes out if I put non-English characters in sp.yaml:

script/server
=> Booting WEBrick...
./script/../config/../lib/localization.rb:62:in `iconv': "叩 aqu鱈 como"
(Iconv::IllegalSequence)
        from ./script/../config/../lib/localization.rb:62:in `load_localized_strings'
        from ./script/../config/../lib/localization.rb:61:in `each'
        from ./script/../config/../lib/localization.rb:61:in `load_localized_strings'
        from ./script/../config/../lib/localization.rb:60:in `open'
        from ./script/../config/../lib/localization.rb:60:in `load_localized_strings'
        from ./script/../config/../lib/localization.rb:52:in `each'
        from ./script/../config/../lib/localization.rb:52:in `load_localized_strings'
        from ./script/../config/../config/environment.rb:59
         ... 8 levels...
        from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in
`require__'
        from /opt/local/lib/ruby/vendor_ruby/1.8/rubygems/custom_require.rb:21:in
`require'
        from /opt/local/lib/ruby/gems/1.8/gems/activesupport-1.2.3/lib/active_support/dependencies.rb:214:in
`require'
        from script/server:3
zmac11:~/pt/TThier/Languages/ruby/j413 oracle$

Notice the msg:
./script/../config/../lib/localization.rb:62:in `iconv': "叩 aqu鱈 como"
(Iconv::IllegalSequence)

It was caused by this line in my sp.yaml:
you_are_here_as: usted está aquí como

Obviously it had problems with
está
and
aquí

I used emacs to create sp.yaml and emacs asked me which character set
encoding I want to use.

I told it utf-8 and emacs saved sp.yaml just fine.

I pulled the actual characters from google and used my mouse to copy
them into emacs:

http://translate.google.com/translate_t

I checked the encoding my browser was using on the google page and it
told me it was using: UTF8
which I'll assume is the same as the utf-8 I'm using in emacs.

Do any of you have some sample localization yaml files which you'd be
willing to share with me?

How about some general troubleshooting ideas?

Thanks,
-Dan

5

This seems to be a yaml / webrick issue though, and not a rails one.

···

On 12/5/05, James Britt <james_b@neurogami.com> wrote:

Dan Bikle wrote:
> People,
>
> I'm trying to use the Localization Generator written by Joe Hosteny:
> http://rubyforge.org/projects/localization/

You will get better, faster, help for Rails at

6

Joe Van Dyk wrote:

···

On 12/5/05, James Britt <james_b@neurogami.com> wrote:

Dan Bikle wrote:

People,

I'm trying to use the Localization Generator written by Joe Hosteny:
http://rubyforge.org/projects/localization/

You will get better, faster, help for Rails at

This seems to be a yaml / webrick issue though, and not a rails one.

My apolgies.

James