CODEWITHSUNDEEP
androidface-detectiongoogle-mlkit
Achint Agrawal
Achint Agrawal

Reputation: 23

MLkit face detection not working with front camera for android

I am using MLKit for face detection on android. I am following the official documentation which can be found here - https://developers.google.com/ml-kit/vision/face-detection/android. I am successfully able to detect faces using the rear camera, but no faces are detected when I switch to the front camera. According to what I have understood from this forum, the problem might be rotation compensation. I have used the same code for this purpose as shown in the documentation -

private static final SparseIntArray ORIENTATIONS = new SparseIntArray();

static {
    ORIENTATIONS.append(Surface.ROTATION_0, 90);
    ORIENTATIONS.append(Surface.ROTATION_90, 0);
    ORIENTATIONS.append(Surface.ROTATION_180, 270);
    ORIENTATIONS.append(Surface.ROTATION_270, 180);
}

/**
 * Get the angle by which an image must be rotated given the device's current
 * orientation.
 */
@RequiresApi(api = Build.VERSION_CODES.LOLLIPOP)
private int getRotationCompensation(String cameraId, Activity activity, Context context)
        throws CameraAccessException {
    // Get the device's current rotation relative to its "native" orientation.
    // Then, from the ORIENTATIONS table, look up the angle the image must be
    // rotated to compensate for the device's rotation.
    int deviceRotation = activity.getWindowManager().getDefaultDisplay().getRotation();
    int rotationCompensation = ORIENTATIONS.get(deviceRotation);

    // On most devices, the sensor orientation is 90 degrees, but for some
    // devices it is 270 degrees. For devices with a sensor orientation of
    // 270, rotate the image an additional 180 ((270 + 270) % 360) degrees.
    CameraManager cameraManager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
    int sensorOrientation = cameraManager
            .getCameraCharacteristics(cameraId)
            .get(CameraCharacteristics.SENSOR_ORIENTATION);
    rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;

    return rotationCompensation;
}

For changing the camera, I just change a global variable cameraID and no other changes are made in the code.

EDIT- In case of front camera, generating input image with rotation = getRotationCompensation(...) - 180 and not subtracting 180 in case of rear camera worked for me. Not sure why that works though. I would still appreciate any explanation regarding this or how this can be avoided.

Upvotes: 1

Views: 2287

Answers (1)

Shiyu
Shiyu

Reputation: 935

The calculation in the above code snippet may not work well for front-facing camera since the image flip.

Could you try to define:

static {
  ORIENTATIONS.append(Surface.ROTATION_0, 0);
  ORIENTATIONS.append(Surface.ROTATION_90, 90);
  ORIENTATIONS.append(Surface.ROTATION_180, 180);
  ORIENTATIONS.append(Surface.ROTATION_270, 270);
}

And use

if (lensFacing == CameraSelector.LENS_FACING_FRONT) {
  rotationCompensation = (sensorOrientation + rotationDegrees) % 360;
} else { // back-facing
  rotationCompensation = (sensorOrientation - rotationDegrees + 360) % 360;
}

to replace the

rotationCompensation = (rotationCompensation + sensorOrientation + 270) % 360;

We will also update the code snippet in the dev doc to reflect that.

Upvotes: 2

Related Questions