CODEWITHSUNDEEP
pythonscrapyscreen-scraping
Caleb90
Caleb90

Reputation: 11

I am trying to Scrape reviews from Amazon using Python 3.x but getting nothing

I am trying to scrape all reviews from amazon. When I ran my code below it returns empty list without any error. I couldn't figure out why. Can you please help me.

from __future__ import unicode_literals

import requests
from scrapy.selector import Selector

def fetch_page(url):
    r = requests.get(url)
    return r.text 

def  review_positive(url):
    #html = fetch_page(url)
    sel = Selector(text = url)
    review =  sel.css(' .a-section review').extract()
    return review


print (review_positive('https://www.amazon.com/Apple-iPhone-Unlocked-GB-Packaging/product-reviews/B01DAJT1AW/ref=cm_cr_arp_d_viewpnt_lft?ie=UTF8&showViewpoints=1&sortBy=helpful&filterByStar=positive&pageNumber=1'))

Upvotes: 1

Views: 806

Answers (1)

Padraic Cunningham
Padraic Cunningham

Reputation: 180401

You should not have spaces in your CSS selector between classes. Instead, use . to separate the classes:

.css('.a-section.review').extract()

Once you fix that, your selector will work fine:

In [6]: rev = response.css('.a-section.review').extract()

In [7]: len(rev)
Out[7]: 10

In [8]: rev = response.css('.a-section review').extract()

In [9]: len(rev)
Out[9]: 0

Upvotes: 1

Related Questions