CODEWITHSUNDEEP
pythonweb-scrapingbeautifulsoup
George Welder
George Welder

Reputation: 4055

Website Scraping with BeautifulSoup : TypeError: 'NoneType' object is not callable

I am an absolute beginner. I try to use BeautifulSoup and scrape a website. I do get the HTML, but I would like to now get all the divs that have the class content_class.

Here is my attempt:

import requests
from BeautifulSoup import BeautifulSoup

#Request the page and parse the HTML
url = 'mywebsite'
response = requests.get(url)
html = response.content

#Beautiful Soup
soup = BeautifulSoup(html)
soup.find_all('div', class_="content_class")

This does not work however. I get:

Traceback (most recent call last): File "scrape.py", line 11, in soup.find_all('div', class_="content_class") TypeError: 'NoneType' object is not callable

What am I doing wrong?

Upvotes: 1

Views: 5563

Answers (2)

Martijn Pieters
Martijn Pieters

Reputation: 1125078

You are using BeautifulSoup version three, but appear to be following the documentation for BeautifulSoup version four. The Element.find_all() method is only available in latest major version (it is called Element.findAll() in version 3).

I strongly urge you to upgrade:

pip install beautifulsoup4

and

from bs4 import BeautifulSoup

Version 3 has stopped receiving updates in 2012; it is severely out of date now.

Upvotes: 3

Logovskii Dmitrii
Logovskii Dmitrii

Reputation: 2973

You're getting this error because there is no method "find_all" in BeautifulSoup, there is "findAll" method, this code should help

 soup.findAll('div', {'class': 'content_class'})

Upvotes: 2

Related Questions