# \[libxml\]: Can't find nodes using XPath, namespaces mess

**URL:** <https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699>\
**Category:** ruby-talk\
**Created:** [31 July 2009 20:33 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699 "2009-07-31T20:33:31Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Stanislaw\_Wozniak](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@Stanislaw\_Wozniak](https://rubytalk.org/u/Stanislaw_Wozniak)\
**Post date:** [31 July 2009 20:33 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/1 "2009-07-31T20:33:31Z")

</div>

Hi,

I am having problems accessing elements in the XML documents using  
XPath. My xml document looks like that:

\<?xml version="1.0" encoding="UTF-8"?\>  
\<configuration-data  
xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance](http://www.w3.org/2001/XMLSchema-instance)";  
xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v1"  
xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
\<attributeList\>  
\<attribute name="siteid" validationRuleName="String" description="Site  
id"\>  
\<tree name="siteid\_hierarchy"\>  
\<treenode name="Root"\>  
\<treenode name="1" /\>  
\</treenode\>  
\</tree\>  
\</attribute\>  
\</attributeList\>  
\</configuration-data\>

My XPath only works when I remove all the namespaces from the root node  
but I do need to access it without modifying the xml.

I am using:  
ruby 1.8.7 (2008-08-11 patchlevel 72) [i386-mswin32]  
libxml-ruby (1.1.3)

> **···**
>
> --  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

---

<div class="post-metadata">

**Author:** ![Matt\_Neuburg](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@Matt\_Neuburg](https://rubytalk.org/u/Matt_Neuburg)\
**Post date:** [1 August 2009 03:00 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/2 "2009-08-01T03:00:07Z")

</div>

Have your run your XML thru a validator? That semicolon looks invalid to  
me. m.

> **···**
>
> Stanislaw Wozniak \<stan@wozniak.com\> wrote:
> 
> > Hi,
> > 
> > I am having problems accessing elements in the XML documents using  
> > XPath. My xml document looks like that:
> > 
> > \<?xml version="1.0" encoding="UTF-8"?\>  
> > \<configuration-data  
> > xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot;](http://www.w3.org/2001/XMLSchema-instance&quot;);  
> > xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v1"  
> > xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
> > \<attributeList\>  
> > \<attribute name="siteid" validationRuleName="String" description="Site  
> > id"\>  
> > \<tree name="siteid\_hierarchy"\>  
> > \<treenode name="Root"\>  
> > \<treenode name="1" /\>  
> > \</treenode\>  
> > \</tree\>  
> > \</attribute\>  
> > \</attributeList\>  
> > \</configuration-data\>
> > 
> > My XPath only works when I remove all the namespaces from the root node  
> > but I do need to access it without modifying the xml.

---

<div class="post-metadata">

**Author:** ![Mark\_Thomas](https://avatars.discourse-cdn.com/v4/letter/m/b5a626/32.png) [@Mark\_Thomas](https://rubytalk.org/u/Mark_Thomas)\
**Post date:** [1 August 2009 03:25 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/3 "2009-08-01T03:25:04Z")

</div>

As Matt said, the document is not well-formed XML. Try adding the  
RECOVER option to the parser, which tells libxml to ignore syntax  
errors like that.

---

<div class="post-metadata">

**Author:** ![Aaron\_Patterson1](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@Aaron\_Patterson1](https://rubytalk.org/u/Aaron_Patterson1)\
**Post date:** [2 August 2009 22:25 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/4 "2009-08-02T22:25:45Z")

</div>

> Hi,
> 
> I am having problems accessing elements in the XML documents using  
> XPath. My xml document looks like that:
> 
> \<?xml version="1.0" encoding="UTF-8"?\>  
> \<configuration-data  
> xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot;](http://www.w3.org/2001/XMLSchema-instance&quot;);  
> xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v1"  
> xmlns="urn:company:platform:foundation:configuration:defn:v1"\>

&nbsp;&nbsp;^^^^^ That says that all nodes inside this document (if not explicitly  
&nbsp;&nbsp;namespaced) belong to an implicit namespace

> \<attributeList\>  
> \<attribute name="siteid" validationRuleName="String" description="Site  
> id"\>  
> \<tree name="siteid\_hierarchy"\>  
> \<treenode name="Root"\>  
> \<treenode name="1" /\>  
> \</treenode\>  
> \</tree\>  
> \</attribute\>  
> \</attributeList\>  
> \</configuration-data\>
> 
> My XPath only works when I remove all the namespaces from the root node  
> but I do need to access it without modifying the xml.

You need to register that namespace with the libxml xpath engine. I'm  
not sure how you register namespaces with libxml-ruby, but with  
nokogiri, I would do this:

&nbsp;&nbsp;doc = Nokogiri::XML(xml)  
&nbsp;&nbsp;doc.xpath('//ns:attribute', 'ns' =\> 'urn:company:platform:foundation:configuration:defn:v1')

Nokogiri will automatically register root level namespaces, so you could  
also do this:

&nbsp;&nbsp;doc = Nokogiri::XML(xml)  
&nbsp;&nbsp;doc.xpath('//xmlns:attribute')

I know there is a way to do this with libxml-ruby, I just don't know the  
syntax off the top of my head. Look through the libxml-ruby  
documentation for "find", and I'm sure you'll find how to register  
namespaces.

> **···**
>
> On Sat, Aug 01, 2009 at 05:33:31AM +0900, Stanislaw Wozniak wrote:
> 
> --  
> Aaron Patterson  
> [http://tenderlovemaking.com/](http://tenderlovemaking.com/)

---

<div class="post-metadata">

**Author:** ![Stanislaw\_Wozniak](https://avatars.discourse-cdn.com/v4/letter/s/e95f7d/32.png) [@Stanislaw\_Wozniak](https://rubytalk.org/u/Stanislaw_Wozniak)\
**Post date:** [1 August 2009 12:01 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/5 "2009-08-01T12:01:58Z")

</div>

Hi, this was a typo, no semicolon in there:

\<?xml version="1.0" encoding="UTF-8"?\>  
\<configuration-data  
xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance](http://www.w3.org/2001/XMLSchema-instance)"  
xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v1"  
xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
\<attributeList\>  
\<attribute name="siteid" validationRuleName="String" description="Site  
id"\>  
\<tree name="siteid\_hierarchy"\>  
\<treenode name="Root"\>  
\<treenode name="1" /\>  
\</treenode\>  
\</tree\>  
\</attribute\>  
\</attributeList\>  
\</configuration-data\>

> **···**
>
> --  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

---

<div class="post-metadata">

**Author:** ![Matt\_Neuburg](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@Matt\_Neuburg](https://rubytalk.org/u/Matt_Neuburg)\
**Post date:** [1 August 2009 15:50 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/6 "2009-08-01T15:50:05Z")

</div>

Then what's the problem? XPath works:

s = \<\<END  
\<?xml version="1.0" encoding="UTF-8"?\>  
\<configuration-data  
xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v  
1"  
xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
\<attributeList\>  
\<attribute name="siteid" validationRuleName="String" description="Site  
id"\>  
\<tree name="siteid\_hierarchy"\>  
\<treenode name="Root"\>  
\<treenode name="1" /\>  
\</treenode\>  
\</tree\>  
\</attribute\>  
\</attributeList\>  
\</configuration-data\>  
END  
require 'rexml/document'  
include REXML  
doc = Document.new(s)  
p XPath.match(doc, "//treenode['Root']/treenode")  
#=\> [\<treenode name='1'/\>]

Oh, wait, you said you were using libxml:

s = \<\<END  
\<?xml version="1.0" encoding="UTF-8"?\>  
\<configuration-data\>  
xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v  
1"  
xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
\<attributeList\>  
\<attribute name="siteid" validationRuleName="String" description="Site  
id"\>  
\<tree name="siteid\_hierarchy"\>  
\<treenode name="Root"\>  
\<treenode name="1" /\>  
\</treenode\>  
\</tree\>  
\</attribute\>  
\</attributeList\>  
\</configuration-data\>  
END  
require 'rubygems'  
require 'xml'  
doc = XML::Document.string(s)  
doc.find("//treenode['Root']/treenode").each do |el|  
&nbsp;&nbsp;p el #=\> \<treenode name="1"/\>  
end

Sorry, I'm failing to guess what problem you're having. Perhaps if you  
showed your actual code? m.

> **···**
>
> Stanislaw Wozniak \<stan@wozniak.com\> wrote:
> 
> > Hi, this was a typo, no semicolon in there:
> > 
> > \<?xml version="1.0" encoding="UTF-8"?\>  
> > \<configuration-data  
> > xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
> > xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v1"  
> > xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
> > \<attributeList\>  
> > \<attribute name="siteid" validationRuleName="String" description="Site  
> > id"\>  
> > \<tree name="siteid\_hierarchy"\>  
> > \<treenode name="Root"\>  
> > \<treenode name="1" /\>  
> > \</treenode\>  
> > \</tree\>  
> > \</attribute\>  
> > \</attributeList\>  
> > \</configuration-data\>

---

<div class="post-metadata">

**Author:** ![Aaron\_Patterson1](https://avatars.discourse-cdn.com/v4/letter/a/e0b2c6/32.png) [@Aaron\_Patterson1](https://rubytalk.org/u/Aaron_Patterson1)\
**Post date:** [2 August 2009 22:39 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/7 "2009-08-02T22:39:29Z")

</div>

> \> Hi, this was a typo, no semicolon in there:  
> \>  
> \> \<?xml version="1.0" encoding="UTF-8"?\>  
> \> \<configuration-data  
> \> xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
> \> xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v1"  
> \> xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
> \> \<attributeList\>  
> \> \<attribute name="siteid" validationRuleName="String" description="Site  
> \> id"\>  
> \> \<tree name="siteid\_hierarchy"\>  
> \> \<treenode name="Root"\>  
> \> \<treenode name="1" /\>  
> \> \</treenode\>  
> \> \</tree\>  
> \> \</attribute\>  
> \> \</attributeList\>  
> \> \</configuration-data\>
> 
> Then what's the problem? XPath works:
> 
> s = \<\<END  
> \<?xml version="1.0" encoding="UTF-8"?\>  
> \<configuration-data  
> xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
> xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v  
> 1"  
> xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
> \<attributeList\>  
> \<attribute name="siteid" validationRuleName="String" description="Site  
> id"\>  
> \<tree name="siteid\_hierarchy"\>  
> \<treenode name="Root"\>  
> \<treenode name="1" /\>  
> \</treenode\>  
> \</tree\>  
> \</attribute\>  
> \</attributeList\>  
> \</configuration-data\>  
> END  
> require 'rexml/document'  
> include REXML  
> doc = Document.new(s)  
> p XPath.match(doc, "//treenode['Root']/treenode")  
> #=\> [\<treenode name='1'/\>]

Wow. These results are just wrong. This is a bug in REXML. In XPath,  
when you do not specify a namespace for your node, that means that you  
want a node \*with no namespace\*.

For example:

&nbsp;&nbsp;require 'rexml/document'  
&nbsp;&nbsp;  
&nbsp;&nbsp;include REXML  
&nbsp;&nbsp;  
&nbsp;&nbsp;s = \<\<END  
&nbsp;&nbsp;\<?xml version="1.0" encoding="UTF-8"?\>  
&nbsp;&nbsp;\<shop\>  
&nbsp;&nbsp;&nbsp;&nbsp;\<!-- car inventory --\>  
&nbsp;&nbsp;&nbsp;&nbsp;\<inventory xmlns="[http://gm.com/&quot;&gt](http://gm.com/&quot;&gt);  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<tire name="all season" /\>  
&nbsp;&nbsp;&nbsp;&nbsp;\</inventory\>  
&nbsp;&nbsp;  
&nbsp;&nbsp;&nbsp;&nbsp;\<!-- bike inventory --\>  
&nbsp;&nbsp;&nbsp;&nbsp;\<inventory xmlns="[Schwinn Bicycles | Schwinn Fitness](http://schwinn.com/&quot;&gt);  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<tire name="street" /\>  
&nbsp;&nbsp;&nbsp;&nbsp;\</inventory\>  
&nbsp;&nbsp;  
&nbsp;&nbsp;&nbsp;&nbsp;\<!-- no namespace inventory --\>  
&nbsp;&nbsp;&nbsp;&nbsp;\<inventory\>  
&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\<tire name="wtf" /\>  
&nbsp;&nbsp;&nbsp;&nbsp;\</inventory\>  
&nbsp;&nbsp;\</shop\>  
&nbsp;&nbsp;END  
&nbsp;&nbsp;  
&nbsp;&nbsp;doc = Document.new(s)  
&nbsp;&nbsp;  
&nbsp;&nbsp;p XPath.match(doc, "//tire")

REXML matches \*all three\* tires. Surely a car tire is not the same as a bike  
tire? Using XPath, how would I query for a tire that has \*no namespace\*  
(the third one) without matching the two that \*do\* belong in a  
namespace (it's possible to do this with REXML, just strange)? The XPath used  
above \*should\* only match the third entry.

This is a broken implementation of XPath.

> Oh, wait, you said you were using libxml:

You have an error in your XML below

> s = \<\<END  
> \<?xml version="1.0" encoding="UTF-8"?\>  
> \<configuration-data\>

&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;^ That "\>" should not be there.  
libxml-ruby has corrections turned on by default, so you've effectively  
removed all namespaces from this document.

> xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
> xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v  
> 1"  
> xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
> \<attributeList\>  
> \<attribute name="siteid" validationRuleName="String" description="Site  
> id"\>  
> \<tree name="siteid\_hierarchy"\>  
> \<treenode name="Root"\>  
> \<treenode name="1" /\>  
> \</treenode\>  
> \</tree\>  
> \</attribute\>  
> \</attributeList\>  
> \</configuration-data\>  
> END  
> require 'rubygems'  
> require 'xml'  
> doc = XML::Document.string(s)  
> doc.find("//treenode['Root']/treenode").each do |el|  
> &nbsp;&nbsp;p el #=\> \<treenode name="1"/\>  
> end
> 
> Sorry, I'm failing to guess what problem you're having. Perhaps if you  
> showed your actual code? m.

Since the namespaces were removed, this example succeeds.

> **···**
>
> On Sun, Aug 02, 2009 at 12:50:05AM +0900, Matt Neuburg wrote:
> 
> > Stanislaw Wozniak \<stan@wozniak.com\> wrote:
> 
> --  
> Aaron Patterson  
> [http://tenderlovemaking.com/](http://tenderlovemaking.com/)

---

<div class="post-metadata">

**Author:** ![Matt\_Neuburg](https://avatars.discourse-cdn.com/v4/letter/m/4af34b/32.png) [@Matt\_Neuburg](https://rubytalk.org/u/Matt_Neuburg)\
**Post date:** [3 August 2009 04:25 UTC](https://rubytalk.org/t/libxml-cant-find-nodes-using-xpath-namespaces-mess/54699/8 "2009-08-03T04:25:04Z")

</div>

Thanks for spotting that. I must have removed the namespace and then put  
it back, to see if I could duplicate the OP's problems, and I must have  
put it back wrong. I wish libxml had just complained that my XML was  
bad...

You're right; fixing the error, I can now duplicate the OP's problem in  
libxml (but not in REXML, as you also observed). And then I can solve  
it:

s = \<\<END  
\<?xml version="1.0" encoding="UTF-8"?\>  
\<configuration-data  
xmlns:xsi="[http://www.w3.org/2001/XMLSchema-instance&quot](http://www.w3.org/2001/XMLSchema-instance&quot);  
xsi:schemaLocation="urn:company:platform:foundation:configuration:defn:v  
1"  
xmlns="urn:company:platform:foundation:configuration:defn:v1"\>  
\<attributeList\>  
\<attribute name="siteid" validationRuleName="String" description="Site  
id"\>  
\<tree name="siteid\_hierarchy"\>  
\<treenode name="Root"\>  
\<treenode name="1" /\>  
\</treenode\>  
\</tree\>  
\</attribute\>  
\</attributeList\>  
\</configuration-data\>  
END  
require 'rubygems'  
require 'xml'  
doc = XML::Document.string(s)  
ns = {"xsi" =\> "urn:company:platform:foundation:configuration:defn:v1"}  
doc.find("//xsi:treenode['Root']/xsi:treenode", ns).each do |el|  
&nbsp;&nbsp;p el #=\> \<treenode name="1"/\>  
end

That is the desired sort of result, I take it. Notice that we register  
the namespace with the XPath engine and that we actually use the  
namescape in our XPath expression. m.

> **···**
>
> Aaron Patterson \<aaron@tenderlovemaking.com\> wrote:
> 
> > You have an error in your XML below