CODEWITHSUNDEEP
androidcomputer-visionandroid-cameraface-detectionandroid-vision
Sira Lam
Sira Lam

Reputation: 5367

Can Mobile Vision API's Face Tracking, not Detection, used independent of its CameraSource?

This question is about using Google's Mobile Vision Face API on Android.

The Story (Background) and what I want to do

I am trying to implement a function that detects faces in a camera view, and overlaying images on those faces.

Now, I have already successfully implemented such feature by using Mobile Vision API's Face Detection. Its mechanism is like this:

  1. A CameraView (I am using Fotoapparat here) that can pass each camera frame in a callback
  2. I turn that frame into a Bitmap
  3. The bitmap is passed to Mobile Vision API for face detection
  4. When Face is detected, I get its position and size
  5. Using that position information, draw something on another custom View.

The problem is, the above process obviously takes too much time. I can only update the overlaying image position 3-5 times every second, even less frequent on slower devices.

By looking at the profiling, the most heavy method is, surprisingly, step 3 (Face detetction). It takes an average of 100ms to perform once.
Second heavy method is converting the NV21 frame into a Bitmap object, which takes around 90ms.
Summing up everything now I get an update FPS of 3~5.

But other than that, every thing works perfectly - image can be captured in high quality, with auto focus and pinch zooming.

How about Face Tracking?

On the other hand, Mobile Vision API provides another API - Face Tracking.

In Google's sample app, the tracking is very fast, it follows almost instantaneously with the faces in the camera preview. As stated in the document, this is because the mechanism is completely different - instead of detecting faces on each frame, once a face is detected, the position simply follows its movement without performing any face detection.

But it fact such mechanism is good enough in my use case!

However, by looking at the sample code, it looks like I have to use its built-in implementation of CameraSource. This can be find in the code below:

    mCameraSource = new CameraSource.Builder(context, detector)
            .setRequestedPreviewSize(640, 480)
            .setFacing(CameraSource.CAMERA_FACING_BACK)
            .setRequestedFps(30.0f)
            .build();

detector is the main character here, and it is only used here when passing to CameraSource. It looks like I have to stick to using this CameraSource.

However, although this camera source has takePicture() method, I cannot find any way to implement auto focus and zooming.

Finally, the question

My ultimate objective is to implement the feature I have mentioned in the beginning, with the below requirements:

  1. High quality image captured
  2. Auto focus
  3. Zoom
  4. Fast face position update (About 10 times in a second is good enough)

1-3 can be done using Face Detection, but not 4;
While 4 can be done using Face Tracker, but not 1-3.

Is there a way to accomplish all 1-4? I welcome any suggestion even if it is to use another library instead of Mobile Vision.

Thanks for reading such long question till the end!

Upvotes: 2

Views: 987

Answers (1)

Alex Cohn
Alex Cohn

Reputation: 57163

CameraSource.java is available on GitHub under a permissive Apache license. Feel free to add auto-focus and zoom.

Upvotes: 2

Related Questions