Manipulate your Webcam stream with OpenCV
Inspired by the background filters of Skype, Teams, Zoom, etc. I was wondering if I could find a way how to apply my own filters to the video stream of my webcam.
I knew that it was pretty easy to open up a python shell and connect to my USB web camera and access its video stream. But how do I forward this to an arbitrary application like Skype, Teams, Zoom, etc. ?
OBS-VirtualCam
Well, I found a solution and its called: OBS-VirtualCam. I have got an Windows 10 PC so I just downloaded the installer and ran it. During installation I was asked if I wanted to have 1 or 4 virtual cameras installed. Since I want to use a single webcam I chose to install 1 virtual cam only.
After that, I opened Skype and went to the audio and video settings. In the top right corner where you can chose your input device and switched to the virtual camera which is apparently displaying an gray image.
Implementation
Now, having the OBS-VirtualCam set up and running let’s start to work on the python code using opencv and pyvirtualcam.
import cv2
import pyvirtualcam
import numpyWe are going to implement one class for reading from the webcam and another class for writing to our virtual camera. Since we want to do this in real-time we are using threads.
class CameraThread:
def __init__(self, cam=0):
self.stream = cv2.VideoCapture(cam)
self.stream.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
self.stream.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)
(_, self.frame) = self.stream.read()
self.stopped = False
def start(self):
Thread(target=self.update, args=()).start()
return self
def stop(self):
self.stopped = True
def update(self):
while True:
if self.stopped:
return
(_, self.frame) = self.stream.read()
def read(self):
return self.frame
def width(self):
return self.frame.shape[1]
def height(self):
return self.frame.shape[0]Let’s go into detail on this piece of code. First we configure the resolution of the VideoCapture. Here you might have to adjust it to your webcam’s driver resolution (it might support multiple). Next, we start a frame reader thread which calls the update method of your CameraThread reading the actual image frame from the input stream.
Having the input stream ready let’s implement a small class which helps to measure FPS before continuing with the virtual camera class.
class FPS:
def __init__(self):
self._n = 100
self._times = np.zeros(self._n)
self._i = 0
def start(self):
self._start = datetime.datetime.now()
def elapsed(self):
self._i = self._i % self._n
self._times[self._i] = (datetime.datetime.now() - self._start).total_seconds()
self._start = datetime.datetime.now()
self._i += 1
return np.mean(self._times) + 0.0000001
def fps(self):
return 1.0 / self.elapsed()
def info(self):
return "{} fps".format(round(self.fps(), 2))Using some point in time as a starting point, this class computes the elapsed time by substracting the current time from the start reference when calling the elapsed method. To convert this to FPS we just need to divide 1 by the elapsed time.
Finally, we need to put everything together. Basically, the VirtualCamera class has one while loop and for each step we take the current available frame from our CameraThread. Then, we can manipulate this frame and write it to the OBS-VirtualCam.
class VirtualCamera:
def __init__(self, cam_id=0, fps=120, mirror_image=False):
self.mirror_image = mirror_image
self.camera = CameraThread(cam_id)
self.measure = FPS()
self.virtual_camera = pyvirtualcam.Camera(width=self.camera.width(),
height=self.camera.height(),
fps=fps)
self.running = False
print("Streaming with resolution: {}x{}".format(self.camera.width(), self.camera.height()))
def run(self):
try:
self.__run__()
except:
pass
finally:
self.stop()
def __run__(self):
self.measure.start()
self.camera.start()
self.running = True
while self.running:
frame = self.camera.read()
frame = cv2.putText(image=frame,
text=self.measure.info(),
org=(80,50),
font=cv2.FONT_HERSHEY_SIMPLEX,
fontScale=1,
color=(240,240,240),
thickness=2,
lineType=cv2.LINE_AA)
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGBA)
if self.mirror_image:
frame = cv2.flip(frame, 1)
self.virtual_camera.send(frame)
self.virtual_camera.sleep_until_next_frame()
def stop(self):
self.running = False
self.camera.stop()
self.virtual_camera.close()In this example I just printed the current FPS to the webcam feed. Also, I enabled the VirtualCam class to mirror the image.
Run your modified webcam
Now it’s time to run what we have implemented so far. Start your program by instancing the VirtualCamera and calling it’s run method.
vcam = VirtualCamera(mirror_image=True)
vcam.run()When you see the following in your shell your virtual camera is running!
Streaming with resolution: 1280x720Now head over to your favorite video conference tool and open the video settings. Select OBS-Camera and you should see your webcam stream with a white FPS measure in the top left corner.
I hope you enjoyed this article. You can find the source code for this in my github repo.
#camera #obs #obs-studio #opencv #virtual #virtualcam #webcam