CODEWITHSUNDEEP
pythonmachine-learningframeworksartificial-intelligenceinterface-builder
Dimitri Finger
Dimitri Finger

Reputation: 19

Use python to identify elements at screen

I want to start a python project thats identify some elements in screen, like a square's size or all elements in screen with a certain color, and i don't even know from where should i start.

For example, all the top points above certain value at the graph.

enter image description here

Anyone could please give me a north?

Upvotes: 0

Views: 1939

Answers (1)

Aziz Sonawalla
Aziz Sonawalla

Reputation: 2502

If you're looking to interact with elements inside a browser, look at Selenium. If you want to control the desktop itself, look into Sikuli.

In either case, you can use OpenCV to identify elements and do template matching.

Edit: more comments after more details from OP

If you're just looking to identify the peaks in the graph, you can take a screenshot of the display at regular intervals using Sikuli or PyScreenshot and then use template matching in OpenCV (either directly or using Sikuli) to get the coordinates of the peaks in the screenshot. The horizontal line across the graph might throw off some of the template matching, but you can play around with the various parameters to get the results you want.

Check out this tutorial for template matching.

Upvotes: 1

Related Questions