CODEWITHSUNDEEP
pythonbeautifulsoup
Jacki
Jacki

Reputation: 21

BeautifulSoup findAll method is not finding all img tags?

I'm writing a web scraper script 

picture = soup.find("div", {"id" : "thumbs-top"})
for link in picture.findAll("img"):
    counter += 1

This counter turns out to be 327 which is clearly not true. I was scraping this imgur gallery : http://imgur.com/a/akHsJ?gallery

Upvotes: 2

Views: 2202

Answers (1)

Imran
Imran

Reputation: 13458

I am getting 575.

>>> import requests
>>> from bs4 import BeautifulSoup
>>> r = requests.get('http://imgur.com/a/akHsJ?gallery')
>>> soup = BeautifulSoup(r.text)
>>> picture = soup.find("div", {"id" : "thumbs-top"})
>>> links = picture.findAll("img")
>>> len(links)
575

Edit: In the comments we determined that the problem is fixed by trying a different html parser with BeautifulSoup. See this question for more info.

>>> soup = BeautifulSoup(r.text, 'html.parser')

Upvotes: 2

Related Questions