CODEWITHSUNDEEP
pythonbeautifulsoup
NaruS
NaruS

Reputation: 168

Web-Scrape Results Not coming Back as Intended

The Issue

No results in the 2nd stage when running

The code

  print("========Stage One========")
  url = url_string
  page = requests.get(url)
  soup2 = BeautifulSoup(page.content, 'html.parser') # Payload 2 soup value

  results = soup2.find(id='list-container')
  print(results.prettify())
  print('========Stage Two========')
  resultSections = results.find_all(class_='mobile-frame pull-left')
  for resultSection in resultSections:
    print(resultSection)
    print("+"*10)

What did I expect/want?

What did I get?

In terms of what I got back after running this script was half of what I expected.

Stage One Worked as intended however Stage Two returned neither results or an error as seen here:

Result Image

What I tried

In terms of what things I tried included the following methods:

results.find_all(div, class_='mobile-frame pull-left') #Current

And

results.find(div, class_='mobile-frame pull-left') # Gives Error

Upvotes: 0

Views: 45

Answers (1)

NaruS
NaruS

Reputation: 168

I seem to have found the solution to my problem.

It appears it gave me the browser emulated page for mobile rather then the computer site.

Refreshing the site gave me the class name of col-md-2 col-sm-3 col-xs-6 which I then used to replace mobile-frame pull-left

Doing this resulted in the desired data.

Upvotes: 1

Related Questions