CODEWITHSUNDEEP
pythonscrapy
D P
D P

Reputation: 31

pumping items out to csv file using python scrapy - issue with how outputted in csv file

Having an issue where I want to add output to a csv file but it does not start below the field name it is placed in the next row in sequence as opposed to placing it at row 2 when populating the playerMins item in the csv file. Can someone please tell me where my code is going wrong?? Here it is:

class EspnSpider3(BaseSpider):
    name = "espn3.org"
    allowed_domains = ["espn3.org"]
    start_urls = [
        "http://scores.espn.go.com/nba/boxscore?gameId=310502004"

    ]

    def parse(self, response):
        hxs = HtmlXPathSelector(response)
        item = EspnItem()
        rows = []
        playerName = []
        playerMins = []

        # player names 
        p_names = hxs.select('(//table[@class="mod-data"][1]/tbody/tr)//a/text()').extract()
        for p_name in p_names:
            print p_name
            yield EspnItem(playerName=p_name)

        # minutes
        p_minutes = hxs.select('(//table[@class="mod-data"][1]/tbody/tr)/td[2]').extract()
        for p_minute in p_minutes:
            print p_minute
            yield EspnItem(playerMins=p_minute)

Upvotes: 1

Views: 1353

Answers (1)

D P
D P

Reputation: 31

Was able to solve my issue, after much googling and rtfm: Trying to Use an ItemExporter in Scrapy

Here is my working code:

def parse(self, response):
    hxs = HtmlXPathSelector(response)
    player_names = hxs.select('(//table[@class="mod-data"][1]/tbody/tr)')
    for p_name in player_names:
        l = XPathItemLoader(item=EspnItem(), selector=p_name )
        l.add_xpath('playerName', 'td[1]/a/text()')
        l.add_xpath('playerMins', 'td[2]')
        yield l.load_item()

Upvotes: 2

Related Questions