CODEWITHSUNDEEP
pythonimagecolors
Kirk
Kirk

Reputation: 159

Process image from screenshot - Python

I'm trying to take a screenshot of a game, bejeweled (an 8x8 board), and extract the board position from the screenshot. I've tried Image/Imagestat, autopy, and grabbing individual pixels from the middle of the slots but these haven't worked. So I'm thinking taking the average value for each square of the 8x8 grid would identify each piece - but I've been unable to do so with Image/Imagestat and autopy.

Anyone know a way to get the pixel or color values for a region of an image? Or a better way to identify segments of an image with a dominant color?

Upvotes: 1

Views: 1188

Answers (1)

Kirk
Kirk

Reputation: 159

I've found a way to do this with PIL using Imagegrab and ImageStat. Here's to grab the screen and crop to the game window:

def getScreen():
    # Grab image and crop it to the desired window. Find pixel borders manually.
    box = (left, top, right, bottom)        
    im = ImageGrab.grab().crop(box)
    #im.save('testcrop.jpg')  # optionally save your crop

    for y in reversed(range(8)):
        for x in reversed(range(8)):
            #sqh,sqw are the height and width of each piece.
            #each pieceim is one of the game piece squares
            piecebox = ( sqw*(x), sqh*(y), sqw*(x+1), sqh*(y+1))
            pieceim = im.crop(piecebox)
            #pieceim.save('piececrop_xy_'+ str(x) + str(y) + '.jpg')

            stats = ImageStat.Stat(pieceim)
            statsmean = stats.mean
            Rows[x][y] = whichpiece(statsmean)

The above creates an image for all 64 pieces, identifies piecetype, and stores that in the array of arrays 'Rows'. I then grabbed the average RGB values with stats.mean for each piecetype and stored them in a dictionary (rgbdict). Copy all the outputs into Excel and filter by color type to get those averages. Then I used an RSS method and that dictionary to statistically match the images to the known piecetypes. (RSS ref: http://www.charlesrcook.com/archive/2010/09/05/creating-a-bejeweled-blitz-bot-in-c.aspx)

rgbdict = {
           'blue':[65.48478993, 149.0030965, 179.4636593],  #1
           'red':[105.3613444,55.95710092, 36.07481793],   #2
           ......
            }
 def whichpiece(statsmean):
        bestScore = 100
        curScore= 0
        pieceColor = 'empty'
        for key in rgbdict.keys():
            curScore = (math.pow( (statsmean[0]/255) - (rgbdict[key][0]/255), 2)
                +  math.pow( (statsmean[1]/255) - (rgbdict[key][1]/255), 2)
                +  math.pow( (statsmean[2]/255) - (rgbdict[key][2]/255), 2) )
            if curScore < bestScore:
                pieceColor = key
                bestScore = curScore  
        return piececolor

With these two functions the screen can be scraped, and the board state transferred into an array upon which moves can be decided. Best of luck if this helps anyone, and let me know if you fine tune a move picker.

Upvotes: 1

Related Questions