CODEWITHSUNDEEP
pythonxmlscrapy
Shiva Krishna Bavandla
Shiva Krishna Bavandla

Reputation: 26648

How to loop over nodes with xmlfeed using scrapy python

Hi i working on scrapy and trying xml feeds first time, below is my code

class TestxmlItemSpider(XMLFeedSpider):
    name = "TestxmlItem"
    allowed_domains = {"http://www.nasinteractive.com"}


    start_urls = [
        "http://www.nasinteractive.com/jobexport/advance/hcantexasexport.xml"
    ]
    iterator = 'iternodes'
    itertag = 'job'


    def parse_node(self, response, node):
        title = node.select('title/text()').extract()
        job_code = node.select('job-code/text()').extract()
        detail_url = node.select('detail-url/text()').extract()
        category = node.select('job-category/text()').extract()

        print title,";;;;;;;;;;;;;;;;;;;;;"
        print job_code,";;;;;;;;;;;;;;;;;;;;;"

        item = TestxmlItem()
        item['title'] = node.select('title/text()').extract()
        .......  
        return item

result:

  File "/usr/lib/python2.7/site-packages/Scrapy-0.14.3-py2.7.egg/scrapy/item.py", line 56, in __setitem__
    (self.__class__.__name__, key))
exceptions.KeyError: 'TestxmlItem does not support field: title'

Totally there are 200+ items so i need to loop over and assign the node text to item but here all the results are displaying at once when we print, actually how can we loop over on nodes in scraping xml files with xmlfeedspider

Upvotes: 1

Views: 377

Answers (1)

Steven Almeroth
Steven Almeroth

Reputation: 8192

From Pablo Hoffman:

You don't have a "title" field declared in your item (TestxmlItem).

You need to add:

title = Field()

Upvotes: 2

Related Questions