WinForms: How to show output of Usb3 Vision webcam

Question

I got a task to show frames from camera as fast as possible. The camera is Basler Dart and it can produce more than 70 frames per second. Unfortunately it does not support stream output, but it produce bitmaps.

Now our solution is similar to the one from example from Basler, but it uses PictureBox to show frames. I've read somewhere that it is slow and better solution should be used. I agree that it is slow because it takes 25% of CPU (only camera, rest of app takes only 5%) when displaying all 70 fps. Unfortunately I haven't found the better solution.

private void OnImageGrabbed(Object sender, ImageGrabbedEventArgs e)
{
  if (InvokeRequired)
  {
    // If called from a different thread, we must use the Invoke method to marshal the call to the proper GUI thread.
    // The grab result will be disposed after the event call. Clone the event arguments for marshaling to the GUI thread.
    BeginInvoke( new EventHandler<ImageGrabbedEventArgs>( OnImageGrabbed ), sender, e.Clone() );
    return;
  }

  try
  {
    // Acquire the image from the camera. Only show the latest image. The camera may acquire images faster than the images can be displayed.

    // Get the grab result.
    IGrabResult grabResult = e.GrabResult;

    // Check if the image can be displayed.
    if (grabResult.IsValid)
    {
      // Reduce the number of displayed images to a reasonable amount if the camera is acquiring images very fast.
      if (!stopWatch.IsRunning || stopWatch.ElapsedMilliseconds > 100)
      {
        stopWatch.Restart();
           
        bool reqBitmapOldDispose=true;

        if(grConvBitmap==null || grConvBitmap.Width!=grabResult.Width || grConvBitmap.Height!=grabResult.Height)
        {
          grConvBitmap = new Bitmap(grabResult.Width, grabResult.Height, PixelFormat.Format32bppRgb);
          grConvRect=new Rectangle(0, 0, grConvBitmap.Width, grConvBitmap.Height);
        }
        else
          reqBitmapOldDispose=false;
           
        // Lock the bits of the bitmap.
        BitmapData bmpData = grConvBitmap.LockBits(grConvRect, ImageLockMode.ReadWrite, grConvBitmap.PixelFormat);
        // Place the pointer to the buffer of the bitmap.
        converter.OutputPixelFormat = PixelType.BGRA8packed;
        IntPtr ptrBmp = bmpData.Scan0;
        converter.Convert( ptrBmp, bmpData.Stride * grConvBitmap.Height, grabResult );
        grConvBitmap.UnlockBits( bmpData );
          
        // Assign a temporary variable to dispose the bitmap after assigning the new bitmap to the display control.
        Bitmap bitmapOld = pictureBox.Image as Bitmap;
        // Provide the display control with the new bitmap. This action automatically updates the display.
        pictureBox.Image = grConvBitmap;
        if (bitmapOld != null && reqBitmapOldDispose)
        {
          // Dispose the bitmap.
          bitmapOld.Dispose();
        }
      }
    }
  }
  catch (Exception exception)
  {
    ShowException(exception, "OnImageGrabbed" );
  }
  finally
  {
    e.DisposeGrabResultIfClone();
  }
}

My idea was to move the load to GPU. I've tryed SharpGL (OpenGL for C#) but it seemed that it cannot consume 70 textures per second but I guess it's because I made the solution in 60 minutes with learning basis of OpenGL.

My question is: What should I use instead of PictureBox to improve power and decrease CPU load? Should I use OpenGL or just limit displayed frames (as it's in example)? The PC has only integrated graphics (Intel i3 6th gen).

Answer 1

Overall when it comes to performance I would recommend doing some profiling to see where the actual bottleneck is. It is much easier to find performance problems if you have some data to go on rather than just guessing at the problem. A good profiler should also tell you a bit about garbage-collection penalty & allocation rates.

Overall I would say the example code looks quite decent:

• There is some rate control, even if the limit of 100ms/10fps looks rather low to me.
• The there does not seem to be much unnecessary copying going on as far as I can see.
• It looks like you are reusing and updating the bitmap rather than recreating it every frame.

Some possible things you could try:

• If the camera is a monochrome you could probably skip the conversion stage, and just do a memory copy from the grab-buffer to the bitmap.
• If the camera is a high resolution model you could consider binning pixels to reduce the resolution of the images.
• We are using WritableBitmap in wpf with fairly good performance. But I'm not sure how it compares to a winforms picturebox.
• You could try doing the drawing yourself, i.e. attach an eventHandler to OnPaint of a panel and use one of the Graphics.DrawImage* methods to draw the latest bitmap. I have no idea if this will make any significant performance difference.
• If the conversion takes any significant time you could try doing this on a background thread. If you do this you might need some way to handle bitmaps in such a way that no bitmap may be accessed from both the worker thread and UI thread at the same time.
• The rate control could probably be improved to "process images as fast the UI thread can handle" instead of a fixed rate.

Another thing to consider is hardware rendering is used. This is normally the case, but there are situations where windows will fallback to software rendering, with a very high CPU usage as a result:

• Servers may lack a GPU at all
• Virtualization might not virtualize the GPU
• Remote desktop might use software rendering