I'm trying to parse a page with Hpricot in order to retrieve a value.
I use Xpather (a firefox extension) in order to get the path of this
value. But when I use this path with Hpricot, it doesn't work. I have
to change it so that it works.
Well, it depends. It'd be helpful to see the page you're working with.
You might want to try asking the Hpricot mailing list as well (To
join: Send a message to hpricot@code.whytheluckystiff.net Cc:
why@whytheluckystiff.net).
Chris
···
On Dec 18, 2007 3:04 PM, Celine <xhanrot@gmail.com> wrote:
Hi all
I'm trying to parse a page with Hpricot in order to retrieve a value.
I use Xpather (a firefox extension) in order to get the path of this
value. But when I use this path with Hpricot, it doesn't work. I have
to change it so that it works.
HPricot doesn't include the whole XPath syntax. You can write a little
function which translates XPath expressions with brackets to HPricot
expressions. I wrote a function for that, but my SVN is right now down
and I can't get it. Drop an answer if you still need it
Firefox includes some missing HTML tags. I ran in it, when I had to
write a little script. <tbody> is added in <table>, probably some more
things, but I didn't find them. You can see the difference when you
download the page with open-uri. Not many pages add <tbody>.
···
On Dec 18, 2007 11:04 PM, Celine <xhanrot@gmail.com> wrote:
Hi all
I'm trying to parse a page with Hpricot in order to retrieve a value.
I use Xpather (a firefox extension) in order to get the path of this
value. But when I use this path with Hpricot, it doesn't work. I have
to change it so that it works.
Hi Chris, thanks for your answer
Here is the page I'm working with : http://finance.yahoo.com
I want to retrieve value of Nasdaq (up left of the page).
I sent the same message on Hpricot ML this afternoon, actually no
answer.
···
On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
Well, it depends. It'd be helpful to see the page you're working with.
You might want to try asking the Hpricot mailing list as well (To
join: Send a message to hpri...@code.whytheluckystiff.net Cc:
w...@whytheluckystiff.net).
I'm very interested in your function. Do you know where I can find
differences between XPath syntax and Hpricot syntax ? What reference
did you use to write your function ?
Celine
···
On Dec 18, 11:25 pm, Thomas Wieczorek <wieczo...@googlemail.com> wrote:
On Dec 18, 2007 11:04 PM, Celine <xhan...@gmail.com> wrote:
> Hi all
> I'm trying to parse a page with Hpricot in order to retrieve a value.
> I use Xpather (a firefox extension) in order to get the path of this
> value. But when I use this path with Hpricot, it doesn't work. I have
> to change it so that it works.
HPricot doesn't include the whole XPath syntax. You can write a little
function which translates XPath expressions with brackets to HPricot
expressions. I wrote a function for that, but my SVN is right now down
and I can't get it. Drop an answer if you still need it
Firefox includes some missing HTML tags. I ran in it, when I had to
write a little script. <tbody> is added in <table>, probably some more
things, but I didn't find them. You can see the difference when you
download the page with open-uri. Not many pages add <tbody>.
>
> Well, it depends. It'd be helpful to see the page you're working with.
> You might want to try asking the Hpricot mailing list as well (To
> join: Send a message to hpri...@code.whytheluckystiff.net Cc:
> w...@whytheluckystiff.net).
>
> Chris
Hi Chris, thanks for your answer
Here is the page I'm working with : http://finance.yahoo.com
I want to retrieve value of Nasdaq (up left of the page).
I see. It's pretty easy to get using element attributes, which
resilient to page changes. If Yahoo decides to add a div in the
hierarchy, or add a new exchange, this shouldn't suddenly fail:
# assuming doc is the Hpricot object for finance.yahoo.com
doc.at('tr[@title="Nasdaq"]/td[2]')
Looking at the page source, the span element that contains the value
actually has an id (yfs_l10_^ixic), but it doesn't look stable, does
it?
I sent the same message on Hpricot ML this afternoon, actually no
answer.
I'm on the Hpricot ML and never saw it. Maybe a hiccup somewhere?
HTH,
Chris
···
On Dec 18, 2007 3:34 PM, Celine <xhanrot@gmail.com> wrote:
On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
I'll post it as soon as the SVN server is up again.
Do you know where I can find
differences between XPath syntax and Hpricot syntax ? What reference
did you use to write your function ?
I used http://code.whytheluckystiff.net/hpricot/wiki/SupportedXpathExpressions
and related pages to get started with it. I found the table/tbody
thing because I didn't get any further with it and thought, that I did
something wrong until I downloaded the page without Firefox using
open-uri.
···
On Dec 18, 2007 11:39 PM, Celine <xhanrot@gmail.com> wrote:
On Dec 18, 11:25 pm, Thomas Wieczorek <wieczo...@googlemail.com> > wrote:
I know it's not nearly as fun as screen-scraping, but you can get the value
for Nasdaq (and many other quotes) on Yahoo! Finance by querying the right
URL for the CSV. The current value can be obtained by fetching:
Just replace [name] with the index or stock you wish to query and each
bracketed date info with integers.
I've replied to a topic before that involved Yahoo! Finance, but it was
specifically about searching for a symbol. Since it's not your case, here's
hoping that directly fetching it will suffice.
···
On Dec 18, 2007 8:34 PM, Celine <xhanrot@gmail.com> wrote:
Hi Chris, thanks for your answer
Here is the page I'm working with : http://finance.yahoo.com
I want to retrieve value of Nasdaq (up left of the page).
I sent the same message on Hpricot ML this afternoon, actually no
answer.
I take that back. That id is almost definitely stable.
doc.at('span[@id="yfs_l10_^ixic"]')
Chris
···
> On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
Looking at the page source, the span element that contains the value
actually has an id (yfs_l10_^ixic), but it doesn't look stable, does
it?
Hi Victor, thank you very much
But, as you said, it isn't very funny, no ?
(but I didn't know that trick, thanks)
···
On 19 déc, 10:53, Vitor Peres <dodecapho...@gmail.com> wrote:
I know it's not nearly as fun as screen-scraping, but you can get the value
for Nasdaq (and many other quotes) on Yahoo! Finance by querying the right
URL for the CSV. The current value can be obtained by fetching:
Just replace [name] with the index or stock you wish to query and each
bracketed date info with integers.
I've replied to a topic before that involved Yahoo! Finance, but it was
specifically about searching for a symbol. Since it's not your case, here's
hoping that directly fetching it will suffice.
Yes, thanks, it runs.
There's something I can't understand : in the Xpath expression I
posted later, when a node has several child DIVs, I access them with
an index (div[2]...), but in Hpricot syntax, DIVs aren't accessed
using an index. So, what trick Hpricot uses to locate "the good" div ?
···
On 18 déc, 23:54, Chris Shea <ch...@ruby.tie-rack.org> wrote:
> > On Dec 18, 11:21 pm, Chris Shea <ch...@ruby.tie-rack.org> wrote:
> Looking at the page source, the span element that contains the value
> actually has an id (yfs_l10_^ixic), but it doesn't look stable, does
> it?
I take that back. That id is almost definitely stable.
On Dec 19, 2007 2:45 PM, Celine <xhanrot@gmail.com> wrote:
Yes, thanks, it runs.
There's something I can't understand : in the Xpath expression I
posted later, when a node has several child DIVs, I access them with
an index (div[2]...), but in Hpricot syntax, DIVs aren't accessed
using an index. So, what trick Hpricot uses to locate "the good" div ?
At some point the path you're using fails. That's why. You could check
node by node, going one level lower each time to see where you start
getting nil from your search. And then you could see what you need to
do to fix the path. That's what I just did:
Tools like Xpather and Firebug can give you paths, but they're not
going to work all the time. But, as I said before, there's a span with
an id attribute that lets you pluck the data without worrying about a
full path, so this is sort of moot.
HTH,
Chris
···
On Dec 19, 2007 4:10 PM, Celine <xhanrot@gmail.com> wrote:
