CODEWITHSUNDEEP
beautifulsoup
leeon
leeon

Reputation: 51

scraping an iframe with beautifulsoup and python

i am trying to scrape the following page:

https://www.dukascopy.com/swiss/english/marketwatch/sentiment/

more exactly, the numbers in the chart. for example, the number 74,19 % in the green bar next to the aud/usd text. i have inspected the elements and found out that the tag for this number is span. but the following code does not return this or any other number in the chart:

import requests
from bs4 import BeautifulSoup
r=requests.get('https://www.dukascopy.com/swiss/english/marketwatch/sentiment/')
soup = BeautifulSoup(r.content, "html.parser")
data = soup('span')
print(data)

Upvotes: 2

Views: 2553

Answers (1)

Grant Herman
Grant Herman

Reputation: 973

So if you incorporate selenium with beautiful soup, you will get all the abilities of selenium to scrape iframes.

try this:

from bs4 import BeautifulSoup
from selenium import webdriver  
from selenium.common.exceptions import NoSuchElementException  
from selenium.webdriver.common.keys import Keys

browser = webdriver.Firefox()
browser.get(bond_iframe)
bond_source = browser.page_source
browser.quit()
soup = BeautifulSoup(bond_source,"html.parser")

for div in soup.findAll('div',attrs={'class':'qs-note-panel'}):
print div

The for loop would be which div tag you are searching for

Upvotes: 1

Related Questions