CODEWITHSUNDEEP
python-3.xseleniumweb-scraping
dreamzboy
dreamzboy

Reputation: 841

Webscrape Interactive Grafana Chart with Mouse Hover Popup Data

I would like to know how to extract the data that popped up when the mouse is hovered over a certain time frame inside a graph panel. Using the demo site as an example, I would like to extract all the web server names and their values at a specific time from https://play.grafana.org/d/000000012/grafana-play-home?orgId=1.

For example:

2020-11-11 15:34:50
web_server_01:  26.75
web_server_02:  55.35
web_server_03:  84.90
web_server_04:  112.95

enter image description here

Selecting the time is not important but the data pop up at a certain time is. There are actually 3 solutions to this but only method 1 or 2 is preferred. Unfortunately, I do not have the know-how to execute both methods.

Method 1 - Use the same datastore API query that Grafana used to pull in the data and plot it on the graph. Grafana does not have an API to extract the data directly as shown.

Method 2 - Use Selenium to webscrape the data that popped up. There was a solution from https://towardsdatascience.com/scraping-interactive-charts-with-python-2bc20a9c7f5c but the difference here is I cannot capture the XPATH of the popup message. It moves with the mouse moves.

Method 3 - Shorten the data time frame, export it to CSV format, and then manipulate the data to the desire result. I would like to import the data directly into the script where possible without downloading.

Upvotes: 0

Views: 987

Answers (2)

Arundeep Chohan
Arundeep Chohan

Reputation: 9969

You'd want the highlighted div tag after clicking on the canvas.

wait = WebDriverWait(driver, 5)
driver.get("https://play.grafana.org/d/000000012/grafana-play-home?orgId=1&from=1612313524334&to=1612316180735")
actions = ActionChains(driver)
canvas = wait.until(EC.presence_of_element_located((By.CLASS_NAME, "flot-overlay")))
actions.move_to_element(canvas).perform()
print(wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "div.graph-tooltip-list-item.graph-tooltip-list-item--highlight"))).text)

Imports

from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait 
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains

Outputs

web_server_03: 98.70

Upvotes: 0

Steve
Steve

Reputation: 66

from selenium.webdriver.common.action_chains import ActionChains

driver.get("https://play.grafana.org/d/000000012/grafana-play-home?orgId=1&from=1612313524334&to=1612316180735")

actions = ActionChains(driver)
canvas=driver.find_element_by_xpath('(//canvas[@class="flot-overlay"])[1]')

actions.move_to_element(canvas).perform()


time.sleep(5)
print(driver.find_element_by_css_selector(
    '[class="graph-tooltip grafana-tooltip"]').text)

use action class and then get the element

Upvotes: 2

Related Questions