CODEWITHSUNDEEP
pythonpython-2.7automated-testsui-automationrobotframework
Deepak Ramakrishnan Kalidass
Deepak Ramakrishnan Kalidass

Reputation: 1702

Recognizing text in a windows/linux GUI in python for automated testing

I am very new to automated testing, and I have some questions. I had a requirement of automating a Windows GUI. I have achieved the button clicks and opening a file using pyautogui and AutoIt. Now i have one task, as I have to detect a word or regex location that is in the screen and click over that.Is there any tool or library in python to achieve that? I have already checked the below items:

Also, I don't want to go with sikuli. Is there any other OCR which can be integrated with Robot Framework?

Upvotes: 4

Views: 4842

Answers (1)

Vasily Ryabov
Vasily Ryabov

Reputation: 10000

For Windows GUI there is powerful Python library pywinauto (can be integrated with Robot Framework as far as I know). It allows some complicated actions like app.Dialog.TreeView.get_item(['1stlevel', '2ndlevel']).expand() and searching controls by regex: app.Dialog.child_window(title_re='^some text - .*$').click().

There is also GUI helper (and code generator) for pywinauto called SWAPY (it's not a record/replay support but very useful). I'm maintaining pywinauto project. More detailed questions are welcome!

For comparison there are several open source UI automation tools (sorted by number of stars at the moment of the answer):

https://github.com/msanders/autopy/ (cross-platform)

https://github.com/Lexikos/AutoHotkey_L/ (not a Python, but popular)

https://github.com/asweigart/pyautogui (cross-platform)

https://github.com/pyatom/pyatom (MAC OS)

https://github.com/ldtp/cobra (cross-platform)

https://github.com/jacexh/pyautoit (AutoIt Python interface)

https://github.com/xcgspring/AXUI

https://github.com/F1ashhimself/UISoup

EDIT: Rating of all known desktop GUI automation tools is maintained every month here: https://github.com/pywinauto/pywinauto/wiki/UI-Automation-tools-ratings

Upvotes: 6

Related Questions