Parse XML that isn't well formed

ruby-talk
1

I have some XML looking like the following, other than being very much
larger (some files are up to 2GB):

<?xml version="1.0" encoding="UTF-8"?>
        <server_url>http://myserver.edu/data/</server_url>
        <server_name>myserver.edu</server_name>
        <uploads>
                <result>
                        <dir>/storage/data/results/</dir>
                        <result_name>hadcm3l_00012_00000118_0</result_name>
                        <file_info>
                        <name>hadcm3l_00012_00000118_0_6.zip</name>
                        <nbytes>5154055</nbytes>
                        <md5_checksum>485600296bb601ab4a3d1d49a9fb1c86</md5_checksum>
                        </file_info>
                        <file_info>
                        <name>hadcm3l_00012_00000118_0_7.zip</name>
                        <nbytes>5153055</nbytes>
                        <md5_checksum>36a600296cb60229a3d1d49a9fb1a10</md5_checksum>
                        </file_info>
                </result>
        </uploads>
</xml>

I've tried a few xml parsers such as xml-simple, libxml and quixml, but
all reject this data as badly formed. One answer would, of course, be
for the data to be re-generated using properly formed xml. Meanwhile, is
there anything that could be done with the existing files? Is it a case
of having to write regexps to parse this sort of thing?

···

--
Posted via http://www.ruby-forum.com/.

2

Note that there should be no </xml> - the line at the top is a
declaration, not an opening tag. Where did </xml> come from? What
happens if you remove that from the data?

-A

···

On 9/19/07, Milo Thurston <knirirr@gmail.com> wrote:

I have some XML looking like the following, other than being very much
larger (some files are up to 2GB):

<?xml version="1.0" encoding="UTF-8"?>
        <server_url>http://myserver.edu/data/&lt;/server\_url&gt;
        <server_name>myserver.edu</server_name>
        <uploads>
                <result>
                        <dir>/storage/data/results/</dir>
                        <result_name>hadcm3l_00012_00000118_0</result_name>
                        <file_info>
                        <name>hadcm3l_00012_00000118_0_6.zip</name>
                        <nbytes>5154055</nbytes>
                        <md5_checksum>485600296bb601ab4a3d1d49a9fb1c86</md5_checksum>
                        </file_info>
                        <file_info>
                        <name>hadcm3l_00012_00000118_0_7.zip</name>
                        <nbytes>5153055</nbytes>
                        <md5_checksum>36a600296cb60229a3d1d49a9fb1a10</md5_checksum>
                        </file_info>
                </result>
        </uploads>
</xml>

3

Alex LeDonne wrote:

Note that there should be no </xml> - the line at the top is a
declaration, not an opening tag. Where did </xml> come from? What
happens if you remove that from the data?

Good point about the XML. Unfortunately, these are the files I have
received and have to deal with them for now.

Removing the final tag gives:

.file.xml:3: parser error : Extra content at the end of the document
<server_name>myserver.edu</server_name>
^
rake aborted!

···

--
Posted via http://www.ruby-forum.com/\.

4

You should have done two things: 1. add root node <server> (with
closing </server> just before </xml>) AND 2. remove the trailing
</xml>

Then it'll be fine.

in your case it's easy:

data.gsub('?>', '?><server>').gsub('</xml>', '</server>')

···

On 9/20/07, Milo Thurston <knirirr@gmail.com> wrote:

Alex LeDonne wrote:
> Note that there should be no </xml> - the line at the top is a
> declaration, not an opening tag. Where did </xml> come from? What
> happens if you remove that from the data?

Good point about the XML. Unfortunately, these are the files I have
received and have to deal with them for now.

Removing the final tag gives:

.file.xml:3: parser error : Extra content at the end of the document
<server_name>myserver.edu</server_name>
^
rake aborted!

5

Jano Svitok wrote:>

You should have done two things: 1. add root node <server> (with
closing </server> just before </xml>) AND 2. remove the trailing
</xml>

Great, thanks.
That should sort out the "legacy" files, and future ones can be
corrected.

I have also been parsing each line with IO.foreach and
/<(.+)[^>]*>(.+?)<(\/.+)>/, which though not as nice as a proper XML
parser does avoid loading huge files into memory in one go.

···

--
Posted via http://www.ruby-forum.com/\.