CODEWITHSUNDEEP
pythonexceptionbeautifulsouptypeerror
user4159592
user4159592

Reputation:

Handling TypeError: 'NoneType' in BeautifulSoup

I am scraping a webpage with BeautifulSoup and the line where I look for a csrf token sometimes throws a TypeError.

Code is:

csrf = soup.find(id="csrfToken-postModuleForm")['value']

The error returned is:

TypeError: 'NoneType' object has no attribute '__getitem__'

I am looking to just continue with the script if that value does not exist instead of having the exception thrown - any idea how that is possible?

Upvotes: 3

Views: 3858

Answers (1)

Jamie Counsell
Jamie Counsell

Reputation: 8103

Something like this

try:
    # Try to get the CSRF token
    csrf = soup.find(id="csrfToken-postModuleForm")['value']
except(TypeError, KeyError) as e:
    # Token not found. Replace 'pass' with additional logic.
    pass

Here, you can add any additional logic, such as: 

    print("CSRF Not Found.")

Make sure you understand how CSRF tokens work. They're an important part of web security.

Upvotes: 3

Related Questions