CODEWITHSUNDEEP
androidandroid-canvassurfaceviewface-detection
Nacho_AlLarre
Nacho_AlLarre

Reputation: 43

Android: How to draw on a Surfaceview which already is displaying a Camara Preview

I am trying to program an application which has to display on the mobile phone's screen what is being filmed by the front camera [The application is not recording/saving anything in the memory of the phone]. Also in case a face is filmed (and detected), it has to appear sourrended by a rectangle.

To do so I'm using:

  1. A Surfaceview to display what is being filmed by the front camera.
  2. A FaceDetectionListener to detect faces in the camera input.

So far the application displays properly what is being filmed by the front camera. It also detects correctly faces. But I'm not able to draw the boundary rectangle around the detected face.

Here are some snippets to show how I'm trying to solve the task.

Activity: 

@SuppressLint("InflateParams")
@TargetApi(Build.VERSION_CODES.ICE_CREAM_SANDWICH)

public class FaceDetectorTutorial extends Activity {

   MySurface mMySurface;
   private SurfaceView surfaceView;

   public void onCreate(Bundle savedInstanceState) {

      super.onCreate(savedInstanceState);
      setContentView(R.layout.activity_face_detector_tutorial);
      setRequestedOrientation(ActivityInfo.SCREEN_ORIENTATION_LANDSCAPE);

      surfaceView = (SurfaceView)findViewById(R.id.camPreview);

      mMySurface = new MySurface(this, surfaceView);

  }     

}

MySurface class:

class MySurface extends SurfaceView implements SurfaceHolder.Callback {  

  Paint paint = new Paint();

  public Camera camera;

  String Tag = "Log: ";


  private SurfaceHolder surfaceHolder;
  boolean preview = false;

 ////////// CLASS CONSTRUCTOR   //////////
 MySurface(Context context, SurfaceView msurfaceView) {
      super(context);

      paint.setColor(Color.RED);
      paint.setStrokeWidth(3);

      surfaceHolder = msurfaceView.getHolder();
      surfaceHolder.addCallback(this);  
      surfaceHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);

 }


 /////////  FACE DETECTION LISTENER /////////
 @SuppressLint("NewApi")
   FaceDetectionListener faceDetectionListener = new FaceDetectionListener(){

        @Override
        public void onFaceDetection(Face[] faces, Camera camera) {

            if (faces.length > 0){

                //Just for the first one detected
                Rect Boundary = faces[0].rect;

                System.out.println(Boundary);

                tryDrawing(Boundary);

            }


 }};//End of FaceDetectionListener

 ////////// SURFACE METHODS ////////
 @Override
 public void surfaceChanged(SurfaceHolder holder, int format, int width,int height) {
         // TODO Auto-generated method stub
        if(preview){
               camera.stopFaceDetection();
               camera.stopPreview();
               preview = false;
         }

       if (camera != null){
               try {                
               camera.setPreviewDisplay(surfaceHolder);
               camera.startPreview();            
               camera.startFaceDetection();
               preview = true;
               } catch (IOException e) {
                   // TODO Auto-generated catch block
                  e.printStackTrace();
               }
           }

  }

  @Override
  public void surfaceCreated(SurfaceHolder holder) {
           // TODO Auto-generated method stub

        int cameraId = -1;      
        int numberOfCameras = camera.getNumberOfCameras();

        for (int i = 0; i < numberOfCameras; i++) {
            CameraInfo info = new CameraInfo();
            Camera.getCameraInfo(i, info);

            if (info.facing == CameraInfo.CAMERA_FACING_FRONT) {
                    cameraId = i;
                    break;
            }
        }

        camera = Camera.open(cameraId);
        camera.setFaceDetectionListener(faceDetectionListener);

  }

  @Override
  public void surfaceDestroyed(SurfaceHolder holder) {
           // TODO Auto-generated method stub
           camera.stopFaceDetection();
           camera.stopPreview();
           camera.release();
           camera = null;
           preview = false;
 }

 private void tryDrawing(Rect Boundary) {
      Log.i(Tag, "Trying to draw...");

      Canvas canvas = surfaceHolder.lockCanvas();

      if (canvas == null) {

          Log.e(Tag, "Cannot draw onto the canvas as it's null");

      } else {

          drawMyStuff(canvas,Boundary);
          surfaceHolder.unlockCanvasAndPost(canvas);
      }
  }

  private void drawMyStuff(final Canvas canvas, Rect Boundary) {

      canvas.drawRect(Boundary.left, Boundary.top, Boundary.right, Boundary.bottom, paint);

      Log.i(Tag, "Drawing...");

  }

}

The Layout:

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    android:layout_width="fill_parent"
    android:layout_height="fill_parent"
    android:orientation="vertical" >

    <SurfaceView
        android:id="@+id/camPreview"
        android:layout_width="fill_parent"
        android:layout_height="wrap_content" />

</LinearLayout>

From the logcat I understand the error is locking the surface. But I don't understand why.

11-23 17:13:51.791: I/System.out(12515): Rect(-187, -495 - 328, 196)
11-23 17:13:51.791: I/Log:(12515): Trying to draw...
11-23 17:13:51.791: E/SurfaceHolder(12515): Exception locking surface
11-23 17:13:51.791: E/SurfaceHolder(12515): java.lang.IllegalArgumentException
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.view.Surface.nativeLockCanvas(Native Method)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.view.Surface.lockCanvas(Surface.java:452)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.view.SurfaceView$4.internalLockCanvas(SurfaceView.java:781)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.view.SurfaceView$4.lockCanvas(SurfaceView.java:757)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at com.example.facedetectiontutorial.MySurface.tryDrawing(FaceDetectorTutorial.java:175)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at com.example.facedetectiontutorial.MySurface.access$0(FaceDetectorTutorial.java:172)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at com.example.facedetectiontutorial.MySurface$1.onFaceDetection(FaceDetectorTutorial.java:109)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.hardware.Camera$EventHandler.handleMessage(Camera.java:815)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.os.Handler.dispatchMessage(Handler.java:99)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.os.Looper.loop(Looper.java:137)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at android.app.ActivityThread.main(ActivityThread.java:5041)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at java.lang.reflect.Method.invokeNative(Native Method)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at java.lang.reflect.Method.invoke(Method.java:511)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:793)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:560)
11-23 17:13:51.791: E/SurfaceHolder(12515):     at dalvik.system.NativeStart.main(Native Method)
11-23 17:13:51.893: E/Log:(12515): Cannot draw onto the canvas as it's null

I based my solution in the accepted answer of the following question:

Android drawing on surfaceview and canvas

But it seems I'm applying it wrong. Could anyone tell me what should I change?

Upvotes: 3

Views: 3359

Answers (2)

fadden
fadden

Reputation: 52313

You can't. A Surface is the producer side of a producer-consumer pair, and there can only be one producer at a time. You can provide camera frames, render in software with Canvas, or render in hardware with OpenGL, but you can't mix them up on a single Surface.

You have a few options. One approach is to use a second SurfaceView that overlaps the first (via a FrameLayout). For this to work, you must specify the Z order for the second Surface -- if you don't, the system will see that you have two Surfaces attempting to occupy the same space, and only one will win. Use the setZOrderMediaOverlay() method to place the new Surface in front of the camera preview but behind the View UI. You also need to change the Surface's color format from the default RGB565 to one that supports transparency; setColorFormat(TRANSLUCENT) or RGB8888 will work. Make sure you erase the Surface to transparent black with an appropriate color transfer mode.

Another approach is to use a custom View. This is more efficient, as it doesn't involve creating a separate Surface (you're drawing on the View UI layer), and Canvas rendering can be hardware-accelerated. The most convenient View to use is the one that's part of SurfaceView -- no change to your layout is required. If your code is sub-classing SurfaceView you're halfway there already.

For an example of multiple overlapping SurfaceViews, see Grafika's "multi-surface test" Activity. If you want to understand more about Android's graphics system, see the architecture doc.

Upvotes: 4

cwbowron
cwbowron

Reputation: 1025

If you using a SurfaceView for displaying the Camera preview, you cannot draw to it. lockCanvas will fail as you've seen.

I would suggest that you use two separate views. One SurfaceView that previews the camera, and a second View (either a SurfaceView or just a custom View), that draws over the top of the preview. You can have both views inside of a FrameLayout or a RelativeLayout.

Upvotes: 1

Related Questions