How to optimize dlib landmark detection?

How to optimize dlib landmark detection? Bitmap 160x120 was processed 7 second. I want to 50 or 100ms. My code:

public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    mRgba = inputFrame.rgba();

    ArrayList<android.graphics.Point> points = new ArrayList();
    try {
        long startTime = System.currentTimeMillis();
        points = LandmarkDetection.getLandmark(matToBitmap(mRgba), this, landmarkPath);
        long endTime = System.currentTimeMillis();
        Log.i(TAG +"Time cost: ", String.valueOf((endTime - startTime) / 1000f) + " sec");
        //drawPoint(points);
        Log.i(TAG, "size = " + String.valueOf(points.size()));
    }catch (Exception e) {
        Log.i(TAG, "bitmap error! " + e.getMessage());
    }
    return mRgba;
}

private Bitmap matToBitmap(@NonNull Mat mat) {
    Bitmap bmp;
    try {
        Mat resized = new Mat();
        Imgproc.resize(mat, resized, new Size(160, 120));
        bmp = Bitmap.createBitmap(resized.cols(), resized.rows(), Bitmap.Config.ARGB_8888);
        Utils.matToBitmap(resized, bmp);
    }catch(Exception e) {
        Log.e(TAG + ":matToBitmap", e.getMessage());
        return null;
    }
    return bmp;
}

And LandmarkDetection class(This method takes all the time):

public static ArrayList<Point> getLandmark(@NonNull Bitmap bmp, Context context, String landmarkPath) {
    mFaceDet = new FaceDet(landmarkPath);
    Log.i(AndroidLauncher.TAG, String.valueOf(new File(context.getExternalCacheDir() + "/shape_predictor_68_face_landmarks.dat").exists()));
    Log.i(AndroidLauncher.TAG, "Ширина: " + String.valueOf(bmp.getWidth()) + "\nВысота: " + String.valueOf(bmp.getHeight()));
    results = mFaceDet.detect(bmp);
    if (results != null) {
        for (final VisionDetRet ret : results) {
            landmarks = ret.getFaceLandmarks();
        }
    }
    return landmarks;
}

What's wrong with my code?

Upvotes: 2

Views: 675

Answers (1)

Evgeniy
Evgeniy

Reputation: 2511

A lot of things can optimize your code:

  1. do not construct face_detector and shape_predictor for every face detection. it can take several seconds. you can use one shape_predictor for all your threads, but face_detector should be one per thread
  2. mFaceDet code is unclear. may be you are resizing image there or doing other operations
  3. http://dlib.net/faq.html#Whyisdlibslow

Upvotes: 1

Related Questions