CODEWITHSUNDEEP
scrapy
trminh89
trminh89

Reputation: 897

Extract text from table with Scrapy

I'm trying to extract Job title insides a table from this page: http://www.chalmers.se/en/about-chalmers/Working-at-Chalmers/Vacancies/Pages/default.aspx

This is the code, but it always returns empty. Any idea how to fix this?

import os
from scrapy.spiders import CrawlSpider
from scrapy.selector import Selector

class mySpider(CrawlSpider):
    name = "myspider"
    allowed_domains = ["www.chalmers.se"]
    start_urls = [  
                  "http://www.chalmers.se/en/about-chalmers/Working-at-Chalmers/Vacancies/Pages/default.aspx",
                  ]

def parse(self, response):
    sel = response.selector

    # try to extract text from a tag inside <td>
    for tr in sel.css("table#jobsTable>tbody>tr"):
        my_title = tr.xpath('td[@class="jobitem"]/a/text()').extract()
        print '================', my_title

I also try to give absolute html path, like bellow but still got empty title:

my_title = response.xpath('/html/body/div/div[1]/div/div[11]/div/table/tbody/tr[1]/td[2]/a/text()').extract()

Upvotes: 0

Views: 62

Answers (1)

gangabass
gangabass

Reputation: 10666

Your website gets above Jobs table from another source (loading it using AJAX call). So you just need to start from another url:

start_urls = ['https://web103.reachmee.com/ext/I003/304/main?site=5&validator=a72aeedd63ec10de71e46f8d91d0d57c&lang=UK&ref=&ihelper=http://www.chalmers.se/en/about-chalmers/Working-at-Chalmers/Vacancies/Pages/default.aspx']

Upvotes: 2

Related Questions