CODEWITHSUNDEEP
pythonhtmlseleniumweb-scrapingphantomjs
VeilEclipse
VeilEclipse

Reputation: 2866

Similar looking screenshots after clicking

I am trying to take screenshot of this page using Selenium. But as Chrome and Firefox does not allow full page screenshot, I am using PhantomJS.

There are 2 sets of durations: 12 Month and Month-to-Month. Hence I am trying to click each tab and take a screenshot.

The code to get the content of the page is:

browser = webdriver.PhantomJS()
browser.set_window_size(1366, 728)
browser.get("http://www.optus.com.au/shop/broadband/mobile-broadband/data-sim-card")
delay = 30 # seconds
try:
    wait = WebDriverWait(browser, delay)
    wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.price")))
    print("\nPage is ready!")
except TimeoutException:
    print("Loading took too much time!")

html = browser.page_source
soup = BeautifulSoup(html, "html.parser")

To get the css classnames of the durations:

durations = soup.body.find("ul", attrs={'class': 'filter-options grouped'})
duration_filtrs = {}
for content in durations.contents:
    duration = content.text  # Storage of the model 64GB, 256GB, 512GB
    css_clss = list(filter(lambda x: x not in ['', 'active'], content.attrs['class']))
    filtr_nm = '.' + '.'.join(css_clss)
    duration_filtrs[duration] = filtr_nm

print(duration_filtrs) 
# {'12 Months': '.filter-option.contract_length_12', 'Month to Month':'.filter-option.contract_length_1'}

To take the screenshot for each duration tab,

for duration, css_cls in duration_filtrs.items():
    browser.find_element_by_css_selector(css_cls).click()
    browser.save_screenshot(duration+'.png')

With the above code, even with slightly different file sizes, I get the similar looking screenshots.

Can someone please tell me what I am doing wrong?

Upvotes: 1

Views: 71

Answers (1)

Buaban
Buaban

Reputation: 5137

I don't know how to fix that issue in PhantomJS. I recommend a workaround with Chrome headless as follow. You just need to specify the window size.

from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support import expected_conditions as EC

chrome_options = Options()
chrome_options.add_argument("--headless")
chrome_options.add_argument("--window-size=2160,3840") # you can adjust the size as you want
browser = webdriver.Chrome(chrome_options=chrome_options)
...
...
for duration, css_cls in duration_filtrs.items():
    button = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,css_cls)))
    browser.save_screenshot('before-'+duration+'.png')
    print(button.text)
    button.click()
    time.sleep(8)
    browser.save_screenshot(duration+'.png')

Upvotes: 2

Related Questions