CODEWITHSUNDEEP
pythonxpathscrapycss-selectors
David Molina
David Molina

Reputation: 161

Python: How I can extract the rank column data using xpath or css selectors?

I create a scrapy spider to extract data from the follow url "https://olympics.com/tokyo-2020/olympic-games/en/results/cycling-road/athlete-profile-n1346266-aalerud-katrine.htm" but I can't to extract data on "rank" column. I've used a for loop to try get data but always I get 'none' value and I'm not understand why. This is part of my code:

class OlympicSpider(scrapy.Spider):
    name = 'ath_spider'
    start_urls = [
        "https://olympics.com/tokyo-2020/olympic-games/en/results/cycling-road/athlete-profile-n1346266-aalerud-katrine.htm"
    ]

    custom_settings = {
        'FEED_FORMAT':'json',                                
        'FEED_URI': 'athletes_tokyo.json' 
    }
    def parse(self, response):
                               
        event = response.css('td > a.eventTagLink::text').getall()
        
        rank=[] 
        for x in range(1,len(event)+1):
            rank.append(response.xpath(
                '//main/div/div[1]/div[1]/div[2]/a[1]/div/table/tbody/tr[%s]/td[3]/text()' %x).get())
                    
        yield{
            'name' : response.css('h1::text').get().strip(),
            'noc' : response.css('div.playerTag::attr(country)').get(),
            'team' : response.css('a.country::text').get(),
            'sport' : response.css('div.container-fluid > div.row > a::text').get(),
            'sex' : response.xpath('//div/div[1]/div[1]/div[2]/div[1]/div/div[2]/div/div[3]/div[1]/div[3]/text()').extract()[-1].strip(),
            'age': response.xpath('//div/div[1]/div[1]/div[2]/div[1]/div/div[2]/div/div[3]/div[1]/div[2]/text()').extract()[-1].strip(), 
            'event':event,
            'rank':rank
        }

Thank you very much in advance

Upvotes: 1

Views: 73

Answers (1)

Prophet
Prophet

Reputation: 33361

The XPath to get the rank values is

//table[@class='table table-schedule']//td[3]/text()

With your specific code it could be something like

for x in range(1,len(event)+1):
    rank.append(response.xpath("(//table[@class='table table-schedule']//td[3])[" + str(x) + "]/text()").get())

Upvotes: 1

Related Questions