OpenCV VideoCapture running on PyGame - repo ref https://github.com/radames/opencv_video_to_pygame

opencv_video_to_pygame.py

from pygame.locals import KEYDOWN, K_ESCAPE, K_q
import pygame
import cv2
import sys

camera = cv2.VideoCapture(1)
pygame.init()
pygame.display.set_caption("OpenCV camera stream on Pygame")
screen = pygame.display.set_mode([1280, 720])

try:
    while True:

        ret, frame = camera.read()

        screen.fill([0, 0, 0])
        frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
        frame = frame.swapaxes(0, 1)
        pygame.surfarray.blit_array(screen, frame)
        pygame.display.update()

        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                sys.exit(0)
            elif event.type == KEYDOWN:
                if event.key == K_ESCAPE or event.key == K_q:
                    sys.exit(0)

except (KeyboardInterrupt, SystemExit):
    pygame.quit()
    cv2.destroyAllWindows()

Thanks for this code! Two comments:

1. I had to change "except KeyboardInterrupt,SystemExit:" to "except (KeyboardInterrupt,SystemExit):" to get this to work.
2. The video frame appears mirror reversed! (I.e., you can't read a picture of English text without using a mirror to view it.) This is not the case when I use OpenCV on its own (without Pygame) to display the video frame, so I don't think it's a problem with my webcam.

frame = cv2.flip(frame, 0) worked for me to unmirror

I know this code is bit old but you can replace the line:

frame = np.rot90(frame)

with:

frame = frame.swapaxes(0,1)

to improve the performance.
Here is the timeit comparision:

python3 -m timeit -s 'import numpy as np' -s 'a = np.zeros((1024,1024,3))' 'a = a.swapaxes(0,1)'
200000 loops, best of 5: 1.2 usec per loop

python3 -m timeit -s 'import numpy as np' -s 'a = np.zeros((1024,1024,3))' 'a = np.rot90(a)'
10000 loops, best of 5: 31.3 usec per loop

The reason for using either of functions is because the opencv represent images in (height, width) format rather than (Width,Height) which is general representation in pygame(or any normal library but not opencv!!!).
So swapaxes basically swaps the two dimensions to obtain (W,H) format
This is similar to how opencv enforces BGR representation and we are explicitly forced to use cvtColor to obtain RGB format.

This little change will also avoid adding extra calls to python cv2.flip(frame,0) in order to get correct result and mirror issues faced by others.

Thanks a lot. This will be useful for me. ;)

@jaxx99 thanks for the optimization! I've also replaced two surfaces step to a more efficient way using pygame.surfarray.blit_array

Folks, I've also create a repo with env for reference https://github.com/radames/opencv_video_to_pygame

I am getting this error:
"ValueError: array must match surface dimensions"
What should I do?

I am getting this error:
"ValueError: array must match surface dimensions"
What should I do?

Change screen size to your camera output size screen = pygame.display.set_mode([640, 480]) worked for me.
You can get size of frame easily E.g: print(frame.shape)

I can't thank you enough for this code. Keep it up.

Guys if anyone had this error

cv2.error: OpenCV(4.5.1) C:\Users\appveyor\AppData\Local\Temp\1\pip-req-build-kh7iq4w7\opencv\modules\imgproc\src\color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function 'cv::cvtColor'

add frame = cv2.VideoCapture(0) below ret, frame = camera.read()

hi @angelman7 can't help much but it could be related to your camera image streaming,
read more https://stackoverflow.com/questions/52676020/opencv-src-empty-in-function-cvtcolor-error

Can you show it the other way too?