CODEWITHSUNDEEP
javaandroidxmlarcoresceneform
arc
arc

Reputation: 31

Set ARCore Sceneview on another screen element

I'm trying to display one arFragment twice on my screen by simply using the feed of one fragment and setting another screen element to the same, but I can't figure out which element to take.

I know, I get the current Camera image by calling

ArFragment arFragment = (ArFragment) getSupportFragmentManager()
                           .findFragmentById(R.id.arFragment);
Image image = arFragment.getArSceneView().getArFrame().acquireCameraImage();

But I don't know how to take another screen object and set the view to the feed, the arFragment gives me. Like, for example:

TextureView secondView = (TextureView) findViewById(R.id.texture);
secondView.setSurfaceTexture((SurfaceTexture) image);

yields an inconvertible types error.

I cannot use another arFragment, because that would have another camera already assigned (one that yields a black screen and a "camera already in use" error, obviously). I haven't found an 

arFrame.assignCamera();

Method, which doesn't matter, since the camera used by the Fragment is only an object, not the real thing. But I can't figure out where the hardware is getting tied to the Fragment. And I can't read nor write there if I recall correctly.

I could convert the feed to a bitmap and maybe place it onto an imageView but I'm a little afraid to do this 60 times a second. There has to be a simple solution, right?...

Can't be so hard to display a view twice -.-

Upvotes: 0

Views: 763

Answers (1)

arc
arc

Reputation: 31

Okay got it. It's a little magic with converting to bmp but guess there's really no direct way.

So I did initialize a bytebuffer, analyze the YUV components of the android.media.image, convert them to a Jpeg, then change it into a Bitmap, rotate it by 90° to match the original picture.

// get the arFragment
arFragment = (ArFragment) getSupportFragmentManager().findFragmentById(R.id.arFragment);
ArSceneView arSceneView = arFragment.getArSceneView();

// set up a Listener to trigger on every frame
arSceneView.getScene().addOnUpdateListener(frameTime -> 
{
  try 
  {
   frame = arSceneView.getArFrame();
   androidMediaImage = frame.acquireCameraImage();
   int imageWidth = androidMediaImage.getWidth();
   int imageHeight = androidMediaImage.getHeight();

   // select the target Container to display the image in
   ImageView secondView = (ImageView) findViewById(R.id.imageView3);
   byte[] nv21;

   // an Android.Media.Image is a YUV-Image which is made out of 3 planes
   ByteBuffer yBuffer = androidMediaImage.getPlanes()[0].getBuffer();
   ByteBuffer uBuffer = androidMediaImage.getPlanes()[1].getBuffer();
   ByteBuffer vBuffer = androidMediaImage.getPlanes()[2].getBuffer();

   // set up a Bytearray with the size of all the planes
   int ySize = yBuffer.remaining();
   int uSize = uBuffer.remaining();
   int vSize = vBuffer.remaining();

   nv21 = new byte[ySize + uSize + vSize];

   // Fill in the array. This code is directly taken from https://www.programcreek.com 
   //where it was pointed out that U and V have to be swapped
   yBuffer.get(nv21, 0 , ySize);
   vBuffer.get(nv21, ySize, vSize);
   vBuffer.get(nv21, ySize + vSize, uSize);

   // combine the three layers to one nv21 image
   YuvImage yuvImage = new YuvImage(nv21, ImageFormat.NV21, imageWidth, imageHeight, null);
   // Open a Bytestream to feed the compressor
   ByteArrayOutputStream out = new ByteArrayOutputStream();
   // compress the yuv image to Jpeg. This is important, because the BitmapFactory can't read a 
   // yuv-coded image directly (belief me I tried -.-)
   yuvImage.compressToJpeg(new Rect(0, 0, imageWidth, imageHeight), 50, out);
   // now write down the bytes of the image into an array
   byte[] imageBytes = out.toByteArray();
   // and build the bitmap using the Factory
   Bitmap bitmapImage = BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);

   // use a Matrix for the rotation
   Matrix rotationMatrix = new Matrix();
   // the thing is basically a bunch of numbers which then can be used to compute the new location of each pixel
   rotationMatrix.postRotate(90);
   // the rotatedImage will be our target image
   Bitmap rotatedImage = Bitmap.createBitmap(bitmapImage, 0,0, bitmapImage.getWidth(), bitmapImage.getHeight(), rotationMatrix, true);

   // it's so easy!!!!
   secondView.setImageBitmap(rotatedImage);
  } catch (NotYetAviableException e) 
   {
     e.printStackTrace();
   }
});

You can obviously correct me if I got it all wrong and there's a way simpler solution to that. But it works at least, so I'm happy <3

Upvotes: 1

Related Questions