CODEWITHSUNDEEP
pythonseleniumselenium-webdriverweb-scrapinghtml5-canvas
Tomás Riquelme Gajardo
Tomás Riquelme Gajardo

Reputation: 55

How to webscrape a canvas element with Python (Selenium)

I need the info from https://childcaredeserts.org/2018/index.html?state=ID, specifically info that pops out when I hover the mouse on a state. I've tried with selenium and python (I'm new at web scraping), but I can't figure out how to get the info of the map, the map appears to be a canvas element (used in HTML5 as have investigated), so I'm a bit confused of how to do this.

Any help is welcome. I'm open to use other programming languages if necessary. Maybe I don't understand very well the canvas element which I'm struggling with.

My code so far:

import time
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600')
driver = webdriver.Chrome('../Downloads/chromedriver', options=options)

driver.get('https://childcaredeserts.org/2018/index.html?state=WA');
action = ActionChains(driver); 
time.sleep(20)

element = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CLASS_NAME, "f1sovyv3")))

#here is where I define the path of info, not sure if it's ok
test = driver.find_elements_by_xpath("//*[name()='div' and @id='root']/*[name()='div']") 

res = []
for el in test:
    hover = action.move_to_element(el) 
    hover.perform() 
    #Below here is supposed to receive the info and append it to a list called res
    #licensed_child_care_providers
    #family_child_care_homes
    #state
    #res.append((licensed_child_care_providers, family_child_care_homes, state)))

for i in res:
    print(i)

driver.quit()

Upvotes: 1

Views: 4930

Answers (1)

Arundeep Chohan
Arundeep Chohan

Reputation: 9969

Here's an example that clicks on the canvas and moves around. Prints the info too.

canvas = driver.find_element_by_css_selector("canvas.mapboxgl-canvas")
action.move_to_element(canvas).click().perform()
box=driver.find_element_by_xpath('//*[@id="root"]/div/div[2]')
print(box.text)
action.move_by_offset(-50, -50).click().perform()
box=driver.find_element_by_xpath('//*[@id="root"]/div/div[2]')
print(box.text)

Outputs

Census Tract 9612
Chelan County
Child Care Desert
Licensed child care providers: 0
Family child care homes: 0
Total child care capacity: 0
Total population: 4682
Population under age 5: 219
Median family income: $60,788
Percent of children with all parents in the labor force: 80%
Maternal labor force participation: 76%
Percent non-Hispanic, white: 69%
Percent non-Hispanic, black/African American: 0%
Percent Hispanic/Latino: 27%
Children per licensed child care slot: No licensed child care providers

And more.

Upvotes: 2

Related Questions