Simon
Simon

Reputation: 31

OpenCv group / locate a number of entities that repeat

I have been struggling with a couple of OpenCV problems I'm trying to overcome. I'm currently working on a game bot that scraps the information off the screen.

It would be great if I could get some pointers or advice on where to go next.

This is the whole screen

whole screen

This is the more interesting part with the stuff highlighted that I'm trying to get. Ive identified a single 'contact' and the important info.

interesting part

With it being a repeating list of picture, name, area, new message notification. I was wondering if I can set a group / layout / template to say when I see a circle(profile Pic) there will be all this interesting stuff x,y pixels to the right?

My other line of thinking would be to find the circle(profile pic) then say offset x,y then cut the image x,y distance to break each 'contact' into a new image then loop through them that way.

ultimately I was aiming to get a array of custom objects that had all this information in that I can then loop over

My second question was the profile pictures themselves. You may notice that some have a little green pills on the foreground. If I screen grab one to do a match template on would the pixels in the profile picture throw the match off?

I was going to do a mask with a circle in it the same height/width but didnt know if the pill symbol itself would then not make a match happen.

I'm currently coding this part up and am constantly hitting brick walls. I can upload what I create and add refinements to it as I go.

Thanks

============================================

EDIT

So this is what I currently have, because I was failing from the beginning I thought it was my approach that was wrong.

import cv2 as cv
import numpy as np
import os
import pyautogui
from time import time
from time import sleep

from windowcapture import WindowCapture

wincap = WindowCapture('Drug Dealer Simulator  ')
screenshot = wincap.get_screenshot()

w,h = screenshot.shape[:2]

screenshot = screenshot[400:h , 0:100]

#convert to gray
screenshot_gray = cv.cvtColor(screenshot, cv.COLOR_BGR2GRAY) 
screenshot_gray = cv.medianBlur(screenshot_gray, 5)
cv.imshow('screenshot_gray', screenshot_gray) 

rows = screenshot_gray.shape[0]
#find the cirles
# detect circles in the image
circles = cv.HoughCircles(screenshot_gray, cv.HOUGH_GRADIENT, 1, 5,param1=100, param2=30,minRadius=1, maxRadius=30)

# ensure at least some circles were found
if circles is not None:
    # convert the (x, y) coordinates and radius of the circles to integers
    circles = np.round(circles[0, :]).astype("int")

    # loop over the (x, y) coordinates and radius of the circles
    for (x, y, r) in circles:
        # draw the circle in the output image, then draw a rectangle
        # corresponding to the center of the circle
        cv.circle(screenshot, (x, y), r, (0, 255, 0), 4)
        cv.rectangle(screenshot, (x - 5, y - 5), (x + 5, y + 5), (0, 128, 255), -1)

    # show the output image
    cv.imshow("output",screenshot)
    cv.waitKey(0)

This was the closest I got with cv2.HoughCircles() and trying to find the circle profile pictures. Im guessing the black background dosent help with some of the images but, I dont know how I can improve that.

enter image description here

I know the pictures are 50x50px so was wondering if I could use a mask of say 70px and cut out a circle and try and match it that way?

Upvotes: 0

Views: 31

Answers (0)

Related Questions