Why CameraX image distorted when saved in Android phone Redmi 9T?

Question

I am using the new device Redmi 9T. Following the basic setup from this post, i manage to up and run an app using CameraX. Everything works fine until i want to save the frame to the local, in the setAnalyzer function

private void bindImageAnalysis(@NonNull ProcessCameraProvider cameraProvider) {
    ImageAnalysis imageAnalysis =
            new ImageAnalysis.Builder().setTargetResolution(new Size(720, 1280))
                    .setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST).build();

    imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this), new ImageAnalysis.Analyzer() {
        @Override
        public void analyze(@NonNull ImageProxy imageProxy) {
            @SuppressLint("UnsafeExperimentalUsageError") Image image = imageProxy.getImage();
            Bitmap finalBitmap = getFinalScaledRotatedBitmap(image,270); //rotate it properly (portrait)
            saveBitmap(finalBitmap, "test"); //save with a dummy name
            imageProxy.close();
        }
    });
    OrientationEventListener orientationEventListener = new OrientationEventListener(this) {
        @Override
        public void onOrientationChanged(int orientation) {
            textView.setText(Integer.toString(orientation));
        }
    };
    orientationEventListener.enable();
    Preview preview = new Preview.Builder().build();
    CameraSelector cameraSelector = new CameraSelector.Builder()
            .requireLensFacing(CameraSelector.LENS_FACING_FRONT).build();
    preview.setSurfaceProvider(previewView.createSurfaceProvider());
    cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector,
            imageAnalysis, preview);
}

 /*
 *  Rotate the image with a rotation degree
 */
 private Bitmap getFinalScaledRotatedBitmap(Image imageData, int viewRotation){
    Bitmap originalBitmap = toBitmap(imageData);
    Matrix matrix = new Matrix();
    matrix.postRotate(viewRotation);
    matrix.postScale(-1.0f, 1.0f);
    Bitmap scaledBitmap = Bitmap.createScaledBitmap(originalBitmap, originalBitmap.getWidth(), originalBitmap.getHeight(), true);
    Bitmap finalRotatedBitmap = Bitmap.createBitmap(scaledBitmap, 0, 0, scaledBitmap.getWidth(), scaledBitmap.getHeight(), matrix, true);
    return finalRotatedBitmap;
}

/*
*  Convert Image format to Bitmap
*/
private Bitmap toBitmap(Image image) {
    Image.Plane[] planes = image.getPlanes();
    ByteBuffer yBuffer = planes[0].getBuffer();
    ByteBuffer uBuffer = planes[1].getBuffer();
    ByteBuffer vBuffer = planes[2].getBuffer();

    int ySize = yBuffer.remaining();
    int uSize = uBuffer.remaining();
    int vSize = vBuffer.remaining();

    byte[] nv21 = new byte[ySize + uSize + vSize];
    //U and V are swapped
    yBuffer.get(nv21, 0, ySize);
    vBuffer.get(nv21, ySize, vSize);
    uBuffer.get(nv21, ySize + vSize, uSize);

    YuvImage yuvImage = new YuvImage(nv21, ImageFormat.NV21, image.getWidth(), image.getHeight(), null);
    ByteArrayOutputStream out = new ByteArrayOutputStream();
    yuvImage.compressToJpeg(new Rect(0, 0, yuvImage.getWidth(), yuvImage.getHeight()), 100, out);

    byte[] imageBytes = out.toByteArray();
    return BitmapFactory.decodeByteArray(imageBytes, 0, imageBytes.length);
}

/*
*  Save Bitmap to local
*/
public void saveBitmap(Bitmap bitmap, String personName) {
    String ROOT =
            Environment.getExternalStorageDirectory().getAbsolutePath() + File.separator + "cameraX";

    File myDir = new File(ROOT + File.separator + personName);
    if (!myDir.mkdirs()) {
        Log.e("FileUtil", "save dir fails");
    }

    String fileName = (new SimpleDateFormat("yyyyMMdd_HHmmss_SSS")).format(Calendar.getInstance().getTime()) + ".jpeg";
    File file = new File(myDir, fileName);
    if (file.exists()) {
        file.delete();
    }

    try {
        FileOutputStream out = new FileOutputStream(file);
        bitmap.compress(Bitmap.CompressFormat.JPEG, 100, out);

        out.flush();
        out.close();
    } catch (Exception var6) {
        Log.e("fileUtil", var6.getMessage());
    }

}

Output? (regardless of Lens front or back)

The thing is, it works fine in other device like honor 8x, or realme. So, what could possibly go wrong?

Answer 1

If you are using camerax library 1.3.0 or later then try

imageProxy.toBitmap()

to get the bitmap and save it. See Android Official Document.

For below 1.3.0

val bitmap = ImageConvertUtils.getInstance().getUpRightBitmap(InputImage.fromMediaImage(image,imageProxy.imageInfo.rotationDegrees))

Also you can try BitmaptUtils.java

BitmaptUtils.getBitmap(imageProxy)

Answer 2

I fixed it using YuvToRgbConverter.kt like this:

class PhotoAnalyzer(
    private val converter: YuvToRgbConverter
) : ImageAnalysis.Analyzer {

     @SuppressLint("UnsafeOptInUsageError")
    override fun analyze(imageProxy: ImageProxy) {
        val bitmapBuffer = Bitmap.createBitmap(imageProxy.width, imageProxy.height, Bitmap.Config.ARGB_8888)

  imageProxy.image?.let { converter.yuvToRgb(it, bitmapBuffer) }
.
.
.

Answer 3

The issue might be with your conversion method toBitmpa(), it assumes the image's format is NV21, but unfortunately not every YUV_888_420 buffer is NV21 format. it can also be NV12, YU12 or YV12 format.

The official CameraX documentation already provides a way to convert YUV images to RGB bitmaps which you should use, it's at the bottom of this section from the documentation.

For sample code that shows how to convert a Media.Image object from YUV_420_888 format to an RGB Bitmap object, see YuvToRgbConverter.kt.