CODEWITHSUNDEEP
automationartificial-intelligence
Poison_Frog
Poison_Frog

Reputation: 175

Can i program an AI that reads what is on the screen and then does some simple task on the computer

So i was thinking is there a way i can program an AI that reads something(numbers mostly, in a font and a particular area on the screen that i will specify) and then perform some clicks on the screen according to what it read...the data(numbers) will change constantly and the AI will have to look out for these changes and act accordingly. I am not asking exactly how do i do that. I am asking whether it is possible and if yes then which approach should i take like for example python or something else and where do i start?

Upvotes: -1

Views: 3036

Answers (2)

gnasher729
gnasher729

Reputation: 52602

It is quite likely that your operating system doesn't allow you access to parts of the screen that are not owned by your application, so you are either blocked at this point, or you are restricted to parts of the screen owned by your application. (If I enter my details on the screen into my banking app, I definitely don't want another app to be able to read it).

Next you'd need to find away to read the pixels on the screen programatically. That will be very different from OS to OS, so very unlikely to be built into your language's library. You might be able to interface with whatever is availabe on your OS, or find a library that does it for you. This will give you an image, made of pixels.

Then you need some OCR software to read the text. AI doesn't seem to be involved in any of this.

Upvotes: -1

Tarik
Tarik

Reputation: 11209

You need a OCR library such as opencv to recognize digits. The rest should be regular programming.

Upvotes: 1

Related Questions