Html method on watir don't return the real html

Hi,
I'm using watir to get the html of a page in order to verify every
single page on w3c.org but what I get with watir is something like:
<HTML lang=is xml:lang="is"
xmlns="http://www.w3.org/1999/xhtml"><HEAD><TITLE>Certus Games</TITLE>
<META http-equiv=Cache-Control content=no-cache>
<META http-equiv=Pragma content=no-cache>
<META http-equiv=Expires content=-1>
<META http-equiv=Content-Type content="text/html; charset=utf-8"><LINK
media=screen href="/CF/css/screen.css" type=text/css
rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css
rel=stylesheet><LINK media=screen href="/CF/css/lib/iestyles.css"
type=text/css rel=stylesheet>
<SCRIPT src="/CF/js/lib/jquery.js" type=text/javascript></SCRIPT>
....

And the real content is:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" lang="is" xml:lang="is">
  <head>
    <meta http-equiv="Cache-Control" content="no-cache"/>
    <meta http-equiv="Pragma" content="no-cache"/>
    <meta http-equiv="Expires" content="-1"/>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"
/>
....

Any idea how to get the real html content?

thank you.

···

--
Posted via http://www.ruby-forum.com/.

Since I think it's impossible to get the real html I'm trying to do it
using net/http library.

In my case I need first to login the user onto the webpage and then go
to the pages I want to get the html.
This is what I'm doing... but it seems to me the user can't login:

    require 'net/http'
    require 'activesupport'
    server="myserver.com"
    port=80

    http = Net::HTTP.new(server, port)
    http.start()

    data={":loginName=>"myLoginName",
      :password=>"myPassord"
    }

    resp,data=http.post("/login.do",data.to_query)
    if resp.code=="200" then
      headers={
        "Cookie"=>resp["set-cookie"].to_s()
      }
    end
    resp,data=http.get('/MyAccount.do',headers)
    html=data
    http.finish()

Any idea???

Thanks.

Mario Ruiz wrote:

···

Hi,
I'm using watir to get the html of a page in order to verify every
single page on w3c.org but what I get with watir is something like:
<HTML lang=is xml:lang="is"
xmlns="http://www.w3.org/1999/xhtml&quot;&gt;&lt;HEAD&gt;&lt;TITLE&gt;Certus Games</TITLE>

--
Posted via http://www.ruby-forum.com/\.

Hi,

Hi,
I'm using watir to get the html of a page in order to verify every
single page on w3c.org but what I get with watir is something like:
<HTML lang=is xml:lang="is"
xmlns="http://www.w3.org/1999/xhtml&quot;&gt;&lt;HEAD&gt;&lt;TITLE&gt;Certus Games</TITLE>
<META http-equiv=Cache-Control content=no-cache>
<META http-equiv=Pragma content=no-cache>
<META http-equiv=Expires content=-1>
<META http-equiv=Content-Type content="text/html; charset=utf-8"><LINK
media=screen href="/CF/css/screen.css" type=text/css
rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css
rel=stylesheet><LINK media=screen href="/CF/css/lib/iestyles.css"
type=text/css rel=stylesheet>
<SCRIPT src="/CF/js/lib/jquery.js" type=text/javascript></SCRIPT>
....

And the real content is:

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd&quot;&gt;

<html xmlns="http://www.w3.org/1999/xhtml&quot; lang="is" xml:lang="is">
<head>
<meta http-equiv="Cache-Control" content="no-cache"/>
<meta http-equiv="Pragma" content="no-cache"/>
<meta http-equiv="Expires" content="-1"/>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"
/>
....

Any idea how to get the real html content?

If you are using Ruby 1.8.6 on Windows, try this code:

require 'win32/api'
require 'watir'

IID_IPersistStreamInit =
[0x7FD52380,0x4E07,0x101B,0xAE,0x2D,0x08,0x00,0x2B,0x2E,0xC7,0x13].pack('LSSC8')

CreateStreamOnHGlobal = Win32::API.new('CreateStreamOnHGlobal', 'LLP',
'L','ole32')
OleSaveToStream = Win32::API.new('OleSaveToStream', 'LL', 'L','ole32')
GetHGlobalFromStream = Win32::API.new('GetHGlobalFromStream', 'LP', 'L','ole32')
GlobalSize = Win32::API.new('GlobalSize', 'L', 'L')
GlobalLock = Win32::API.new('GlobalLock', 'L', 'L')
CopyMemory = Win32::API.new('RtlMoveMemory', 'PLL', 'V')
GlobalUnlock = Win32::API.new('GlobalUnlock', 'L', 'L')

browser = Watir::Browser.new
browser.goto("http://www.google.com")

ptr = browser.document.inspect.scan(/:(0x[\da-f]+)?>/).to_s.hex
p = 0.chr * 4
CopyMemory.call(p,ptr+16,4)
data = p.unpack('L').first
p = 0.chr * 4
CopyMemory.call(p,data,4)
dispatch = p.unpack('L').first

lpVtbl = 0.chr * 4
table = 0.chr * 28
CopyMemory.call(lpVtbl,dispatch,4)
CopyMemory.call(table,lpVtbl.unpack('L').first,28)
table = table.unpack('L*')
queryInterface = Win32::API::Function.new(table[0],'PPP','L')
p = 0.chr * 4
hr = queryInterface.call(dispatch,IID_IPersistStreamInit,p)
persiststream = p.unpack('L').first

p = 0.chr * 4
CreateStreamOnHGlobal.call(0, 1, p)
stream = p.unpack('L').first
if OleSaveToStream.call(persiststream,stream) == 0
  p = 0.chr * 4
  GetHGlobalFromStream.call(stream,p)
  h = p.unpack('L').first
  size = GlobalSize.call(h)
  buf = 0.chr * size
  ptr = GlobalLock.call(h)
  if ptr != 0
    CopyMemory.call(buf,ptr,size)
    GlobalUnlock.call(ptr)
    html = buf[16..-1]
  else
    html = nil
  end
end

puts html

Regards,
Park Heesob

···

2009/5/19 Mario Ruiz <tcblues@gmail.com>:

It's working!!!!! Thanks a lot.

Since I'm not using the last Watir version I changed Watir::Browser.new
for Watir::IE.new and it's working fine.

Again... thanks a lot.

Heesob Park wrote:

···

Hi,

2009/5/19 Mario Ruiz <tcblues@gmail.com>:

rel=stylesheet><LINK media=print href="/CF/css/print.css" type=text/css

Any idea how to get the real html content?

--
Posted via http://www.ruby-forum.com/\.