CODEWITHSUNDEEP
iosmultithreadingavfoundationgrand-central-dispatchgpuimage
Gizmodo
Gizmodo

Reputation: 3222

iOS AVFoundation and Multithreading

In every sample code of Apple's AVFoundation demonstrations, a seperate queue for all the session tasks.

dispatch_queue_t sessionQueue = dispatch_queue_create("session queue", DISPATCH_QUEUE_SERIAL);

This is also used for observing focus changes, exposure changes, whitebalance changes, ect:

- (void)viewWillAppear:(BOOL)animated
{
     [super viewWillAppear:animated];
     dispatch_async([self sessionQueue], ^{
          [self addObservers];
          [[self session] startRunning];
     });
}

- (void)viewDidDisappear:(BOOL)animated
{
     dispatch_async([self sessionQueue], ^{
          [[self session] stopRunning];
          [self removeObservers];
     });
     [super viewDidDisappear:animated];

}

When I followed this, everything was smooth, until I adopted 'GPUImage', a 3rd party api. I was informed that multi-threading or seperate queueing is not needed for this api since it's all taken care of within. So I took the 'sessionQueue' completely out.

However, now my app runs really choppy, and I think it's due to main queue having told to handle most branched-off tasks.

Can I have a seperate queue just for observing self.videoCamera changes? Would that take majority of the load off of mainQueue?

Upvotes: 2

Views: 3352

Answers (1)

matt
matt

Reputation: 534925

Hang on, there, gizmodo! I didn't say (I think you mean me) that "multi-threading or separate queueing is not needed". I said I wasn't convinced it was needed in the (tiny) code you actually showed in the earlier question.

And you are also perfectly right that Apple directly advises starting the capture session on a background thread. As their comment in the AVCamManual sample code explicitly says:

-[AVCaptureSession startRunning] is a blocking call which can take a long time. We dispatch session setup to the sessionQueue so that the main queue isn't blocked (which keeps the UI responsive).

So far, so good. But what I also said was, and I stand by this: multithreading is hard.

Let's just take the code in this new question as an example. You are saying:

 dispatch_async([self sessionQueue], ^{
      [[self session] stopRunning];
      [self removeObservers];
 });

Now, that might be an okay thing to say, and then again it might not. In Apple's sample code, having decided to start the capture session on a background thread, they are careful to touch that capture session only on that thread (queue) from then on. And you seem to be doing the same. That's good.

But that isn't all you do in your background thread. How do I know what happens in your removeObservers method, for example? If it touches any properties of self or any other object, or if it calls any methods that do, or if it calls any part of Cocoa that is not thread-safe (which, in fact, is probably most of Cocoa), then you are headed for serious trouble down the road when things mysteriously and undebuggably go wrong - if not for you, then (even worse) for some user. (And of course I am omitting, as being simply assumed, the danger of doing anything connected in any way with the app's interface on any but the main thread.)

And how do I know what else you do in other parts of your code? Apple is careful to touch [self session] only in this one background thread; that is an important rule, and is part of what good multithreading involves. But are you aware of this rule? Do you ever make the mistake of touching [self session] on the main thread or some other thread different from this one? How do I know? I don't.

So, to sum up once again, use multithreading all you like, provided you know what you are doing and provided you are prepared to take the consequences. Multithreading is hard. (Ooops, I said it again.)

Now, perhaps you have thoroughly vetted your removeObservers method (and all the other methods you are calling from background threads here). I am guessing that removeObservers probably copies directly the model in Apple's sample code, in which case everything is probably just fine. Perhaps you are an awesome multithreader and everything I'm saying here is well known to you and just washes off like water from a duck's back. That's great! Then these concerns are not an issue to you, and you should just disregard my words. In short, you can and probably should use a background thread for dealing with your capture session, just so long as you know what you are doing and don't step outside the bounds of safety. If you know what those bounds are, great! My warning was, and is, just in case you do not know.

Upvotes: 10

Related Questions