CODEWITHSUNDEEP
androidcamerasurfaceviewaugmented-reality
r.markiew
r.markiew

Reputation: 41

Can I use lockCanvas() in onPreviewFrame callback?

I'm fighting with an IllegalArgumentException when using lockCanvas() in onPreviewFrame.

Basically what I want to achieve is to grab the frame, process it, and draw it directly to the surface of SurfacePreview.

here is my code of onPreviewFrame

@Override
public void onPreviewFrame(byte[] data, Camera camera) {
    Canvas canvas = null;

    if (mHolder == null) {
        return;
    }

    int mImgFormat = mCamera.getParameters().getPreviewFormat();

    try {
        canvas = mHolder.lockCanvas();
        canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
        mHolder.unlockCanvasAndPost(canvas);
    } catch (Exception e) {
        e.printStackTrace();
    } finally {
        if (canvas != null)
        {
            mHolder.unlockCanvasAndPost(canvas);
        }
    }
}

I have read a lot of documentation and topics about camera in android, and I suppose that locking the surface that frames are drawn onto isn't possible, because it's beyond scope of the application control. Is it true?

One of possible solutions is to draw on top of the surface view with another view, but I want to keep my processed frames (there will be some canny edge detection and color corrections, which may be time consuming) drawn in sync with preview. When camera will keep spitting frames with good fps my calculations will have a hard time catching up, and in worst case scenario draw an overlay that is dozens of frames behind.

I studied openCV source and for me it looks like they've managed to do this in CameraBridgeViewBase.java:

protected void deliverAndDrawFrame(CvCameraViewFrame frame) {
    Mat modified;

    if (mListener != null) {
        modified = mListener.onCameraFrame(frame);
    } else {
        modified = frame.rgba();
    }

    boolean bmpValid = true;
    if (modified != null) {
        try {
            Utils.matToBitmap(modified, mCacheBitmap);
        } catch(Exception e) {
            Log.e(TAG, "Mat type: " + modified);
            Log.e(TAG, "Bitmap type: " + mCacheBitmap.getWidth() + "*" + mCacheBitmap.getHeight());
            Log.e(TAG, "Utils.matToBitmap() throws an exception: " + e.getMessage());
            bmpValid = false;
        }
    }

    if (bmpValid && mCacheBitmap != null) {
        Canvas canvas = getHolder().lockCanvas();
        if (canvas != null) {
            canvas.drawColor(0, android.graphics.PorterDuff.Mode.CLEAR);
            canvas.drawBitmap(mCacheBitmap, (canvas.getWidth() - mCacheBitmap.getWidth()) / 2, (canvas.getHeight() - mCacheBitmap.getHeight()) / 2, null);
            if (mFpsMeter != null) {
                mFpsMeter.measure();
                mFpsMeter.draw(canvas, 20, 30);
            }
            getHolder().unlockCanvasAndPost(canvas);
        }
    }
}

I'm either missing something or I'm just too old for this :)

Also, it's my first post here so hello everyone :)

Upvotes: 4

Views: 1363

Answers (1)

user2672441
user2672441

Reputation: 21

Actually, openCV's camera plays a little trick. In the CameraBridgeViewBase.java and JavaCameraView.java file. I found that a SurfaceTexture class(from api 11) is used to receive frames from the camera. The advantage of the SurfaceTexture is you do not need to always draw it on the screen(it can be contained in the memory). Then a SurfaceView class is responsible to draw the processed frame from the onPreviewFrame function. Note this SurfaceView is not bound to the camera. Hope this can be help.

Upvotes: 1

Related Questions