CODEWITHSUNDEEP
htmlwebscrapyweb-crawlerscreen-scraping
terminatorash2199
terminatorash2199

Reputation: 305

Scrapy query is returning an empty list

I want to scrape a website to the links. https://www.rentomojo.com/mumbai/furniture/bedroom-furniture-on-rent enter image description here

The link is the href link inside div. My scrapy code is 

response.css("div.col-xs-6 col-sm-4 col-mgbtm a::attr(href)").extract()

but this does not work.

I even tried using xpath

response.xpath("//div[@class='col-xs-6 col-sm-4 col-mgbtm']/a/@href").extract()

But this also does not work.

Any help would be appreciated.

Upvotes: 1

Views: 274

Answers (2)

Ikram Khan Niazi
Ikram Khan Niazi

Reputation: 809

Simply write response.css(".col-xs-6 ::attr(href)").extract()

No need to write HTML tags within the selector when the class or id is present. Similarly no need to write a tag ::attr(href) is enough for extracting links.

Upvotes: 1

Manualmsdos
Manualmsdos

Reputation: 1545

I tried your xpath code in bash:

scrapy shell https://www.rentomojo.com/mumbai/furniture/bedroom-furniture-on-rent
response.xpath("//div[@class='col-xs-6 col-sm-4 col-mgbtm']/a/@href").extract()

and it work fine.

Code with css

response.css("div.col-xs-6 col-sm-4 col-mgbtm a::attr(href)").extract()

return nothing.

Upvotes: 1

Related Questions