CODEWITHSUNDEEP
pythonlxml
diemacht
diemacht

Reputation: 2102

How to parse broken HTML with LXML

I'm trying to parse a broken HTML with LXML parser on python 2.5 and 2.7

Unlike in LXML documentation (http://lxml.de/parsing.html#parsing-html) parsing a broken HTML does not work:

from lxml import etree
import StringIO
broken_html = "<html><head><title>test<body><h1>page title</h3>"
parser = etree.HTMLParser()
tree   = etree.parse(StringIO.StringIO(broken_html))

Result:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "lxml.etree.pyx", line 2954, in lxml.etree.parse (src/lxml/lxml.etree.c:56220)
  File "parser.pxi", line 1550, in lxml.etree._parseDocument (src/lxml/lxml.etree.c:82482)
  File "parser.pxi", line 1578, in lxml.etree._parseMemoryDocument (src/lxml/lxml.etree.c:82764)
  File "parser.pxi", line 1457, in lxml.etree._parseDoc (src/lxml/lxml.etree.c:81562)
  File "parser.pxi", line 965, in lxml.etree._BaseParser._parseDoc (src/lxml/lxml.etree.c:78232)
  File "parser.pxi", line 569, in lxml.etree._ParserContext._handleParseResultDoc (src/lxml/lxml.etree.c:74488)
  File "parser.pxi", line 650, in lxml.etree._handleParseResult (src/lxml/lxml.etree.c:75379)
  File "parser.pxi", line 590, in lxml.etree._raiseParseError (src/lxml/lxml.etree.c:74712)
lxml.etree.XMLSyntaxError: Opening and ending tag mismatch: h1 line 1 and h3, line 1, column 50

Upvotes: 27

Views: 13761

Answers (3)

Jerome Anthony
Jerome Anthony

Reputation: 8021

lxml allows you load a broken xml by creating a parser instance with recover=True

etree.HTMLParser(recover=True)

You could use the same technique when creating the parser.

Upvotes: 12

user2665694
user2665694

Reputation:

You might try to use lxml.html instead

>>> import lxml.html
>>> broken_html = "<html><head><title>test<body><h1>page title</h3>"
>>> root = lxml.html.fromstring(broken_html)
>>> lxml.html.tostring(root)
'<html><head><title>test</title></head><body><h1>page title</h1></body></html>'

Upvotes: 9

Fred Foo
Fred Foo

Reputation: 363597

Don't just construct that parser, use it (as per the example you link to):

>>> tree = etree.parse(StringIO.StringIO(broken_html), parser=parser)
>>> tree
<lxml.etree._ElementTree object at 0x2fd8e60>

Or use lxml.html as a shortcut:

>>> from lxml import html
>>> broken_html = "<html><head><title>test<body><h1>page title</h3>"
>>> html.fromstring(broken_html)
<Element html at 0x2dde650>

Upvotes: 28

Related Questions