CODEWITHSUNDEEP
pythonopencvpygamewebcam
Adam McNeilly
Adam McNeilly

Reputation: 81

display cv2.VideoCapture image inside Pygame surface

I'm trying to use opencv (cv2) to stream a webcam feed into a pygame surface object. The problem is the colors aren't displaying correctly. I think it is the type casting, but I'm having trouble understanding the pygame surface documentation to know what it expects.

This code demonstrates what I'm talking about

import pygame
from pygame.locals import *
import cv2
import numpy

color=False#True#False
camera_index = 0
camera=cv2.VideoCapture(camera_index)
camera.set(3,640)
camera.set(4,480)

#This shows an image the way it should be
cv2.namedWindow("w1",cv2.CV_WINDOW_AUTOSIZE)
retval,frame=camera.read()
if not color:
    frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
cv2.flip(frame,1,frame)#mirror the image
cv2.imshow("w1",frame)

#This shows an image weirdly...
screen_width, screen_height = 640, 480
screen=pygame.display.set_mode((screen_width,screen_height))

def getCamFrame(color,camera):
    retval,frame=camera.read()
    if not color:
        frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
    frame=numpy.rot90(frame)
    frame=pygame.surfarray.make_surface(frame) #I think the color error lies in this line?
    return frame

def blitCamFrame(frame,screen):
    screen.blit(frame,(0,0))
    return screen

screen.fill(0) #set pygame screen to black
frame=getCamFrame(color,camera)
screen=blitCamFrame(frame,screen)
pygame.display.flip()

running=True
while running:
    for event in pygame.event.get(): #process events since last loop cycle
        if event.type == KEYDOWN:
            running=False
pygame.quit()
cv2.destroyAllWindows()

The ultimate goal I have is to create a small photo booth application for a DIY wedding next year. I'm new to programming, but I've managed to cobble this together. I was also trying to accomplish this with VideoCapture, which outputs a PIL, which I also couldn't get to work with the surface object. I want to use a pygame surface so I can animate and overlay count-down text, borders, etc.

Update: The issue was that the cv2 function camera.read() returns a BGR image, but the pygame.surfarray expects a RGB image. This is fixed with the line

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

Also, when converting to grayscale, the following code works:

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
frame=cv2.cvtColor(frame,cv2.COLOR_GRAY2RGB)

So, the function getCamFrame should now be

def getCamFrame(color,camera):
    retval,frame=camera.read()
    frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)
    if not color:
        frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)
        frame=cv2.cvtColor(frame,cv2.COLOR_GRAY2RGB)
    frame=numpy.rot90(frame)
    frame=pygame.surfarray.make_surface(frame)
return frame

Upvotes: 8

Views: 11274

Answers (3)

Mario Abbruscato
Mario Abbruscato

Reputation: 819

This worked for me

to adapt it in your code ...

windowSurface = screen  #or use directly variable screen

    # convert windowSurface to cv2 ------------------------------------
    view = pygame.surfarray.array3d(windowSurface)
    #  convert from (width, height, channel) to (height, width, channel)
    view = view.transpose([1, 0, 2])
    #  convert from rgb to bgr
    img_bgr = cv2.cvtColor(view, cv2.COLOR_RGB2BGR)
    cv2.imshow('windowname',img_bgr)
    cv2.waitKey(10)

Upvotes: 0

Juan
Juan

Reputation: 11

I tried your code, but I'm only getting a picture not a movie, so i copied 

frame=getCamFrame(color,camera)
screen=blitCamFrame(frame,screen)
pygame.display.flip()

into a while loop, it worked but then the video was flipped, to fix it i added cv2.flip(frame,1,frame) # mirror the image before frame=numpy.rot90(frame) in getcamFrame function and now everything works fine.

Sorry for the poor English.

Upvotes: 1

Transformer
Transformer

Reputation: 3760

No the color error lies in here

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2GRAY)

so for normal screen color u simply change that to 

frame=cv2.cvtColor(frame,cv2.COLOR_BGR2RGB)

That will do because it works for me

Upvotes: 2

Related Questions