CODEWITHSUNDEEP
pythonpython-3.xbeautifulsouptypeerror
Abhay Salvi
Abhay Salvi

Reputation: 1129

My Beautiful Soup code is returning TypeError: 'NoneType' object is not iterable

I am using Beautiful Soup for web scraping and getting a TypeError here.

My code is as follows :

import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.amazon.com/s?k=asus&rh=n%3A300189&nav_sdd=aps&pd_rd_r=58b28d7d-1955-433b-b33b-b1b5dcf1f522&pd_rd_w=MJzan&pd_rd_wg=QG3cj&pf_rd_p=6d81377b-6d6c-4363-ae02-8fa202ed7b50&pf_rd_r=X0BDDAPN7TTW0ZT1REX6&qid=1583290662&ref=sxwds-sbc_c2")
soup = BeautifulSoup(r.text, 'html.parser')
x = soup.find(Class='a-size-medium a-color-base a-text-normal')

for vari in x:
    print(vari.get_text())

The error:

Traceback (most recent call last):
  File "c:/Users/intel/Desktop/Untitled-1.py", line 8, in <module>
    for vari in x:
TypeError: 'NoneType' object is not iterable

I don't think my class id is wrong...

Upvotes: 0

Views: 291

Answers (1)

Gonzalo Hernandez
Gonzalo Hernandez

Reputation: 737

Your code doesn't work because Amazon is blocking your automated request.

import requests
from bs4 import BeautifulSoup

r = requests.get("https://www.amazon.com/s?k=asus&rh=n%3A300189&nav_sdd=aps&pd_rd_r=58b28d7d-1955-433b-b33b-b1b5dcf1f522&pd_rd_w=MJzan&pd_rd_wg=QG3cj&pf_rd_p=6d81377b-6d6c-4363-ae02-8fa202ed7b50&pf_rd_r=X0BDDAPN7TTW0ZT1REX6&qid=1583290662&ref=sxwds-sbc_c2")
soup = BeautifulSoup(r.text, 'html.parser')
with open("out.html", "w") as f:
   f.write(str(soup))

out.html

Upvotes: 2

Related Questions