CODEWITHSUNDEEP
pythoniframexpathamazon-web-services
brain storm
brain storm

Reputation: 31262

how to extract reviews from iframeurl returned by amazon api in python?

I am trying to get the text content of reviews of a given product in amazon using its api. But I am not able to work it out. Here is what I have:

result = api.item_lookup('B00062B6QY', ResponseGroup='Reviews',
     TruncateReviewsAt=256, IncludeReviewsSummary=False)
iframeurl=result.xpath('//*[local-name()="IFrameURL"]/text()')[0].strip()
print iframeurl
reviews=requests.get(iframeurl)
reviews.raise_for_status()
#data = json.loads(reviews.text)
root = ET.fromstring(reviews.text)
print root

The output is:

http://www.amazon.com/reviews/iframe?akid=helloworld&alinkCode=xm2&asin=B00062B6QY&atag=welcomehome-20&exp=2014-01-28T19%3A06%3A20Z&summary=0&truncate=256&v=2&sig=HIDDEN%3D
Traceback (most recent call last):
  File "amazon_api_new.py", line 36, in <module>
    root = ET.fromstring(reviews.text)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1300, in XML
    parser.feed(text)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
xml.etree.ElementTree.ParseError: mismatched tag: line 867, column 2

PS: I have changed the iframeurl printed out just to clear the api key details

EDIT: imageenter image description here from firebug

Upvotes: 0

Views: 783

Answers (1)

Guy Gavriely
Guy Gavriely

Reputation: 11396

instead of using ElementTree, try to load reviews.text to lxml like:

>>> from lxml import etree
>>> parser = etree.HTMLParser()
>>> tree   = etree.parse(StringIO(reviews.text), parser)

>>> result = etree.tostring(tree.getroot(),
...                         pretty_print=True, method="html")
>>> print(result)
...

of course, you can then use lxml xpath for further parsing

Upvotes: 1

Related Questions