Is it possible to use face_landmark.tflite from Mediapipe to generate face mesh in Android independently?

Question

I'm working on a face tracking app (Android studio / Java) and I need to identify face landmarks. I'm interested using Mediapipe face mesh model. The problem is: I use Windows OS, and Mediapipe is not working on Windows OS.

I have very basic knowledge in Tensorflow, Can anybody explain to me how can i use Mediapipe's face_landmark.tflite model to detect faces in images and generate face mesh in Android studio with Java independently without the whole Mediapipe framework?

Answer 1

You can try look into my notebook below for usage example in python. This only needs tflite model and does not require Mediapipe installation.

https://nbviewer.jupyter.org/github/quickgrid/CodeLab/blob/master/tensorflow/MediaPipe_Face_Mesh_TFLite_Python_Direct_Inference.ipynb

This is the output image,

This should give a starting point to use android tflite interpreter to get face landmarks and draw them. It will require a face detector such as blazeface to output the face bounding box first.

As I have not implemented this model in android yet I cannot say what else may be needed. Further details may be found in mediapipe face mesh codes. The notebook is based on this code,

MediaPipe TensorflowLite Iris Model

https://github.com/shortcipher3/stackoverflow/blob/master/mediapipe_iris_2d_landmarks.ipynb

Further references,

https://github.com/google/mediapipe/blob/master/mediapipe/modules/face_landmark/face_landmark.tflite

https://google.github.io/mediapipe/solutions/face_mesh

Model card with input, output details,

https://drive.google.com/file/d/1QvwWNfFoweGVjsXF3DXzcrCnz-mx-Lha/view

Alternate option

Android ML Kit also has face landmarks with very good documentation and code example.

https://developers.google.com/ml-kit/vision/face-detection

https://developers.google.com/android/reference/com/google/mlkit/vision/face/package-summary