Reputation: 184
I'm working on an app where I need to overlay a webcam video of people on a green/blue background screen (shades of green/blue may differ) with a still background image or video.
Unfortunately, I can't find any professional solution (in the form of C# framework/library) to solve chroma key image filtering issue.
I decided to write my own solution using the OpenCV library. Using @Burak's solution for Python, I partially solved the issue by porting the code to C#:
private void Worker_DoWork(object sender, DoWorkEventArgs e)
{
Mat mask, result = null;
var worker = (BackgroundWorker)sender;
while (!worker.CancellationPending)
{
using (var frame = videoCap.RetrieveMat())
{
// Get a chroma key mask
mask = ChromaKeyMask(frame);
if (videoBack != null)
{
if (videoBack.Width != frame.Width && videoBack.Height != frame.Height)
{
videoBack = videoBack.Resize(frame.Size());
}
result = videoBack.Clone();
frame.CvtColor(ColorConversionCodes.RGB2BGR).CopyTo(result, mask);
}
Dispatcher.Invoke(() =>
{
FrameImage.Source = frame.ToWriteableBitmap();
MaskImage.Source = mask.ToWriteableBitmap();
ResultImage.Source = result?.CvtColor(ColorConversionCodes.BGR2RGB).ToWriteableBitmap();
mask?.Dispose();
result?.Dispose();
});
}
Thread.Sleep(20);
}
}
private Mat ChromaKeyMask(Mat src)
{
Mat mask;
using (var hsvMat = src.CvtColor(ColorConversionCodes.BGR2HSV))
{
var min = new Scalar(minH, minS, minV);
var max = new Scalar(maxH, maxS, maxV);
// Threshold the HSV image to extract green color
mask = hsvMat.InRange(min, max);
Cv2.BitwiseNot(mask, mask);
// Smooth edges of the mask
var size = new OpenCvSharp.Size(3, 3);
var kernel = Cv2.GetStructuringElement(MorphShapes.Rect, size);
Cv2.MorphologyEx(mask, mask, MorphTypes.Open, kernel);
Cv2.GaussianBlur(mask, mask, size, 0, 0);
}
return mask;
}
This code is working but still not good enough for production grade: mask is "unstable" and "floating", and final image, after combining with background, has a green "halo".
So, my questions are:
mask = hsvMat.InRange(min, max);
?I'm really sorry (I'm new to OpenCV and image processing), but I'm asking for answers with specific working solutions, not general thoughts about chroma key (yes, I've already read the Wikipedia article and watched a lot of questions/answers here on StackOverflow) . There is no need to use C#; Python and C++ will work fine.
Here you can find a modified @Barak Python script with images for test.
I tried a different SDKs from https://www.visioforge.com and https://www.graphicsmill.com: pretty unstable and/or simple non-working.
I need a professional grade chroma key filtering solution, to process webcam stream in realtime.
Upvotes: 0
Views: 363