CODEWITHSUNDEEP
pythonopencvgitlab-ci-runnerpyautoguipyscreeze
IG2000
IG2000

Reputation: 1

Issues with element detection on remote machine using PyAutoGUI and OpenCV

Given: PyAutoGUI and OpenCV. Their specific task is to find a particular element on the current screen by exact match. This is done by taking a screenshot of the entire screen and then searching for an image (rectangular, with clear edges) on it. If successful, the coordinates of where the match was found are returned.

The problem: Locally everything works fine; the code runs, and all desired elements are found. Apart from the local environment, there is a test machine with a GitLab runner that executes the code. When running on this machine, some elements are found with a 100% success rate, while others are not found under any conditions, regardless of the image being searched for, whether it's larger or smaller. It feels like it can't find elements associated with additional pop-up windows within the application. It's unclear what role this plays because the screenshot captures all these windows, so they should be found.

Attempts made that didn't help: Set the same resolution on both machines, converted images to the same color space, used pixel fonts to reduce noise, tried several different libraries and search methods—always failing at the same stage. Checked libraries and path variables—everything is the same.

What can be done to understand the problem and how to fix it?

Upvotes: 0

Views: 146

Answers (1)

IG2000
IG2000

Reputation: 1

Well, everything was solved in a rather non-trivial way. The problem was in the program under test and the settings of the test user. The point is that the UI of the program adapted to the DPI of the user's screen, and it did it only once - for the first user who logged in after rebooting, so the layout of interface elements and font thickness changed, everywhere a little bit, but as a result it led to a big difference. Even though fonts and screen resolution were standardized. I took over this project from another developer (whose dpi was 2 times different from mine) - his DPI settings were preserved on the test machine and test profile. This was not easy to trace, as I was on the test machine under my own user with normal settings, and the tests were run using the test user. However, @Dan Mašek's reply helped me to some extent to organize the solution approach and decompose the problem, thank you.

Upvotes: 0

Related Questions