CODEWITHSUNDEEP
c#videouwp
Rachel
Rachel

Reputation: 33

How to get SoftwareBitmap from MediaPlayer in UWP

I am currently working on a project where I am receiving a live video stream from an IP camera as a MediaPlayer object. The end goal is to be able to use Windows.Media.OCR to extract text from a frame every second or so, and for this I need a SoftwareBitmap.

From the Microsoft UWP documentation, it appears that the frame can be taken from the MediaPlayer object with the method CopyFrameToVideoSurface(CanvasBitmap). I can create a CanvasBitmap from a SoftwareBitmap, but I have not found a way to create a SoftwareBitmap from the CanvasBitmap without having to save the file (which I am trying to avoid, I do not need to retain the images). I am hoping I am missing something simple, is there a way to get a SoftwareBitmap from a MediaPlayer object?

I have been referencing this example for using MediaPlayer in frame server mode. I do not need to display the image, so if possible it seems best to avoid the CanvasBitmap if I can.

MediaPlayer

private async Task GetStream()
{
    mediaPlayer = new MediaPlayer()
    {
        Source = MediaSource.CreateFromStream(placeholder, "video")
    };

    mediaPlayer.VideoFrameAvailable += VideoFrameAvailable;
    mediaPlayer.IsVideoFrameServerEnabled = true;
    mediaPlayer.Play();
}

private async void VideoFrameAvailable(MediaPlayer sender, object args)
{
    // Get frame from media player, create SoftwareBitmap
    await ExtractText(softwareBitmapImg);
}

My code for the OCR portion is relatively simple and works like a charm when I have a SoftwareBitmap to provide.

OCR

private async Task ExtractText()
{
    Language ocrLanguage = new Language("en-us");
    OcrEngine ocrEngine = OcrEngine.TryCreateFromLanguage(ocrLanguage);

    var ocrResult = await ocrEngine.RecognizeAsync(bitmap);
    String text = ocrResult.Text;
}

Upvotes: 3

Views: 1555

Answers (1)

Breeze Liu - MSFT
Breeze Liu - MSFT

Reputation: 3808

The CanvasBitmap implement the IDirect3DSurface interface, Whenever the VideoFrameAvailable handler is called, the CopyFrameToVideoSurface method is used to copy the contents of the frame to an IDirect3DSurface. We need the CanvasBitmap object to copies the current frame from the MediaPlayer into the CanvasBitmap when the CopyFrameToVideoSurface is called, but you do not need to display the image.

private async void mediaPlayer_VideoFrameAvailable(MediaPlayer sender, object args)
{
    CanvasDevice canvasDevice = CanvasDevice.GetSharedDevice();
    await Dispatcher.RunAsync(Windows.UI.Core.CoreDispatcherPriority.Normal, async () =>
    {
        SoftwareBitmap softwareBitmapImg;
        SoftwareBitmap frameServerDest = new SoftwareBitmap(BitmapPixelFormat.Rgba8, 100, 100, BitmapAlphaMode.Premultiplied);

        using (CanvasBitmap canvasBitmap = CanvasBitmap.CreateFromSoftwareBitmap(canvasDevice, frameServerDest))
        {
            sender.CopyFrameToVideoSurface(canvasBitmap);

            softwareBitmapImg = await SoftwareBitmap.CreateCopyFromSurfaceAsync(canvasBitmap);

        }
        await ExtractText(softwareBitmapImg);
    });
}

Upvotes: 3

Related Questions