Hi all.
I am very new to Ruby (5 days old) so my question might sound very
noobish. I am posting it only cause I couldn't find a solution.
I am using ruby to scrape content of a site.
To be precise I am having problems with the ’ character.
Sample source page :
http://thainews.prd.go.th/newsenglish/previewnews.php?news_id=255108040023
The encoding is in tis-620 and I use Iconv to convert it to utf8,
however the special quote character gives the following error on iconv
/home/....../main.rb:37:in `iconv': "\222s announcement "...
(Iconv::IllegalSequence)
the affected code area
body = story.search("//font[@color=\"#333333\"]").inner_html
body = body.gsub(/<(.|\n)+?>/, "")
body = body.gsub(/�/, "\'")
puts body
body = Iconv.iconv("utf8", "tis-620", body) #<-- this is line 37
puts body
Or try the following on irb
require 'rubygems'
require 'net/http'
require 'open-uri'
require 'iconv'
story =
Hpricot(open('http://thainews.prd.go.th/newsenglish/previewnews.php?news_id=255108040023'
))
body = story.search("//font[@color=\"#333333\"]").inner_html
body = body.gsub(/<(.|\n)+?>/, "")
body = body.gsub(/’/, "\'")
puts body
no matter whatever i put in the "’" it doesn't replace anything and the
iconv still gives errors.
I am looking for pointers on one of the following.
1) how do i replace "’" to "'" ?
or 2) How can I make iconv ignore the "’" ?
At first I thought this to be a I18n issue, but i guess getting rid of
the special character would be a simple string manipulation which i dont
get.
···
--
Posted via http://www.ruby-forum.com/.
