CODEWITHSUNDEEP
c#imagewinformscameraframe-rate
Daniel M
Daniel M

Reputation: 105

How to save save high frame rate camera images with live preview

I have two high speed USB3 cameras (Ximea) and want to code an application for image recording. Framerates are up to 500fps at VGA resolution but I also want to use the 2Mpx resolution at 170fps. Their .Net SDK tells me that I should simply "get" the images in a loop. My problem is that I have no idea how to get the images and save them while still showing a live preview. Everytime I add some code to actually update the picturebox the frame rate drops drastically.

At the moment I utilize a recording function that is called with 

Task.Run(() => Record());

and inside the Record() I have a loop getting the bitmaps

while(record == true)
{
    Camera.GetImage(out myImage, timeout); //From Ximea .Net SDK
    Info = Camera.GetLastImageParams();
    Timestamp = Info.GetTimeStamp();
    ThreadPool.QueueUserWorkItem(state => SaveImage(myImage, filepath, Timestamp));
}

with the SaveImage being

private void SaveImage(Bitmap myImage, string filepath, double Timestamp)
{
    try
    {
        lock(myImage)
        {
             myImage.Save(filepath + Timestamp.ToString("0.00000") + ".tif");
        }
    }
    catch{}
}

How can I show a live preview while recording and how can I make the entire code more stable (at the moment there are some dropped frames because of "object already in use"-errors or "generic error in GDI+" at the Image.Save() call, that I skip with the try/catch statement)?

Upvotes: 0

Views: 1520

Answers (2)

keelan
keelan

Reputation: 11

I believe you can tell the Ximea API how many image buffers you want in the incoming queue... use XI_PRM_BUFFER_POLICY and XI_PRM_BUFFERS_QUEUE_SIZE appropriately to make that queue length somewhat long. Then, have a thread that, when activated, copies an image out of a XI_IMG struct into your own buffer. Activate that thread every n frames (based on the size of the Ximea image buffer queue size)... but don't do any memory copies in the loop that actually calls xiGetImage. You should probably block in your thread to avoid tearing (because the Ximea code could get back around to using the same buffer again if you're not fast enough at copying the data out)... but you could then dynamically adjust the number of buffers so you can finish your copy within the time you have. Also, you may consider copying the image data to ANOTHER buffer if you're doing something that takes a long time...

pseudo-code (sorry, it's C-ish):

// sync objects and a global image buffer pointer
CRITICAL_SECTION cs;
void *buf;
HANDLE ev;

int CopyImageThreadProc(...)
{
  while (true)
  {
    if (WaitOnSingleObject(ev) == WAIT_OBJ_0)
    {
      EnterCriticalSection(cs);
      // copy the image data at buf where ever you want
      LeaveCriticalSection(cs);
    }
  }
}

int main(...)
{
  // set up ximea api with appropriate buffering

  // create event and critsec, start thread

  while (!done)
  {
    XI_IMG img;
    xiGetImage(dev, 10, &img);

    // every 15 frames, tell your thread to go...
    // if you find that the critsec is causing a hiccup, you can adjust this
    // but remember to adjust the queue length, too
    // if you change this to TRY entercriticalsection, you can determine that
    if ((img.acq_nframe % 15) == 0)
    {
      EnterCriticalSection(cs);
      buf = img.bp;
      SetEvent(ev);
      LeaveCriticalSection(cs);
    }
  }

  // clean up
}

Upvotes: 1

Chris
Chris

Reputation: 5514

Add each captured frame to a queue, then have a worker thread that takes those images, one at a time, and saves them. Trying to write multiple images to disk at the same time will most likely be slower. Also, always Dispose of any GDI objects or you will run into trouble really fast. I would think not doing that is what is giving you the exceptions.

As for showing the images, make sure you are not trying to display every image. Your monitor most likely runs at 60 Hz, so anything faster than that will be a waste. I also suspect that (with the performance of GDI), you won't necessarily be able to achieve even that. So I suggest you have a second queue with images to display, and if you see that queue getting too large, your program will need to slow down a bit and not push as many frames to the queue.

Edit: And of course, as @Franck mentions, if your disk can't keep up, your queue/buffer will fill up quickly. Compressing the images might help, assuming they have suitable content for compression and that your processor can keep up.

Edit: What you need is a producer-consumer pattern. There are many ways of doing this, but one might be something like:

// blocking collection
private BlockingCollection<Bitmap> m_Queue = ...

// camera thread
while( run )
{
    var bitmap = GrabFrame();
    m_Queue.Add( bitmap );
}

// worker thread
try
{        
    while( true )
    {
        // Take() will block if the queue is empty
        var bitmap = m_Queue.Take();
        bitmap.Save( ... );
        bitmap.Dispose();
    }
catch( InvalidOperationException )
{
     // you'll end up here if you call `m_Queue.CompleteAdding()`
     // (after the queue has been emptied, of course)
}

As for displaying the images, you could probably use something similar, with the addition of having some code that determines if it's time to push a new image or not.

Upvotes: 0

Related Questions