Camera landscape preview in portrait mode

Question

I have a dialog design where camera preview should be landscape in portrait mode (see pics). I choose preview size correctly through getOptimalPreviewSize() and when I don't use setDisplayOrientation(90) I get this:

When I use setDisplayOrientation(90) I get this:

Does anybody have any ideas how to fix it? Can android do that kind of thing?

Answer (thanks to Eddy Talvala):
Due to you can't process the whole cam picture in portrait when your cam view in xml is horizontal, you should crop. I decided to fit horizontally and crop all that in the bottom:

|           |
| _________ |   _____
||         ||  |     |
||         ||  | ^-^ |
||  ______ ||  | \_/ |
|| | ^-^  |||  |  |  |    
|| |_\_/_ |||  |__|__|
||   image ||   sensor
||         ||
||         ||
||_________||
||  < O [] ||
|___________|
    device

You should use TextureView to specify matrix for fit and cropping. Something like that (kotlin):

fun camInit() {
    val camParam = camera.parameters
    val optimalSize = getOptimalPortraitPreviewSize(camParam.supportedPreviewSizes, width, height)

    camParam.setPreviewSize(optimalSize.width, optimalSize.height)
    camera.parameters = camParam

    camera.setPreviewTexture(surface)

    val scaleDelta = optimalSize.height - preview.width // for portrait
    val scaleY: Float = (optimalSize.width.toFloat() - scaleDelta) / preview.height // for portrait

    val matrix = Matrix()
    matrix.setScale(1f, scaleY)    // 1f cause we fit horizontally

    preview.setTransform(matrix)

    camera.startPreview()
}

Notice that for portrait mode you should use specific getOptimalProtraitPreviewSize(), cause standard func that everybody uses to get camera preview optimal size (with ASPECT_TOLERANCE and other things) can return you a size with a small resolution. Try something like this:

fun getOptimalPortraitPreviewSize(sizes: List<Camera.Size>, previewWidth: Int) {

    var selectedSize: Camera.Size = sizes[0]

    for (size in sizes) {
        // use size's height cause of portrait
        val currentSizeWidthDelta = abs(size.height - previewWidth)
        val selectedSizeWidthDelta = abs(selectedSize.height - previewWidth)
        if (currentSizeWidthDelta < selectedSizeWidthDelta) selectedSize = size
    }

    return selectedSize
}

Answer 1

Thanks @Eddy Talvala and @blinker! I bring my function to correctly update the sizes in any orientation:

  public void invalidateSize(TextureView textureView) {
        Point cameraResolution = getCameraResolution();
        int previewRotation = getPreviewRotation();

        // Camera and preview on screen in the opposite orientation
        if (previewRotation == 90 || previewRotation == 270) {
            //noinspection SuspiciousNameCombination
            cameraResolution = new Point(cameraResolution.y, cameraResolution.x);
        }

        float cameraAspectRatio = cameraResolution.x / (float) cameraResolution.y;
        float textureAspectRatio = textureView.getWidth() / (float) textureView.getHeight();

        float scaleX = cameraAspectRatio / textureAspectRatio;
        float scaleY = textureAspectRatio / cameraAspectRatio;

        if (scaleX > scaleY) {
            scaleY = 1;
        } else {
            scaleX = 1;
        }

        Matrix matrix = new Matrix();
        matrix.setScale(scaleX, scaleY,
                textureView.getWidth() / 2f, textureView.getHeight() / 2f);

        textureView.setTransform(matrix);
}

Answer 2

If you are using the Camera2 API. When get mPreviewSize by ChooseOptimalSize you can swap the width and height. (The width and height is from your view). And then

textureView.setAspectRatio(mPreviewSize.Height, mPreviewSize.Width);

When create preview session

texture.SetDefaultBufferSize(mPreviewSize.Width, mPreviewSize.Height);

Then you will preview the normal image

Answer 3

The long edge of the image sensor lines up with the long edge of the device.

(bad ascii art):
____________
|           |
| _________ |   _____
||         ||  |     |
||         ||  |     |
||         ||  |     |
||         ||  |     |
||         ||  |_____|
||         ||   sensor
||         ||   physical 
||         ||   orientation
||_________||
||  < O [] ||
|___________|
    device

Given that, there's no way you can draw the image like this:

_____________
|           |
| _________ |   _____
||         ||  |     |
||         ||  |     |
||  ______ ||  |     |
|| |      |||  |     |
|| |______|||  |_____|
||   image ||   sensor
||         ||
||         ||
||_________||
||  < O [] ||
|___________|
    device

unless you either rotate the image (in which case up isn't up):

_____________
|           |
| _________ |   _____
||         ||  |     |
||         ||  | ^-^ |
||  ______ ||  | \_/ |
|| | <-\__|||  |  |  |    
|| |_<-/__|||  |__|__|
||   image ||   sensor
||         ||
||         ||
||_________||
||  < O [] ||
|___________|
    device

or you stretch the image horizontally, which looks pretty bad:

_____________
|           |
| _________ |   _____
||         ||  |     |
||         ||  | ^-^ |
||  ______ ||  | \_/ |
|| | ^___^|||  |  |  |    
|| |___|__|||  |__|__|
||   image ||   sensor
||         ||
||         ||
||_________||
||  < O [] ||
|___________|
    device

or you crop a slice of the image, which reduces the FOV a lot:

_____________
|           |
| _________ |   _____
||         ||  |     |
||         ||  | ^-^ |
||  ______ ||  | \_/ |
|| | ^-^  |||  |  |  |    
|| |_\_/__|||  |__|__|
||   image ||   sensor
||         ||
||         ||
||_________||
||  < O [] ||
|___________|
    device

Because the image sensor is landscape when the phone is landscape, there's no way to place a landscape preview in a portrait UI without one of those three things happening. You either need a portrait preview in a portrait UI, or you need to crop the image down. That's not a limitation of Android, it's just a limitation of geometry.

If you want to crop, you'll probably want to send the camera data to a SurfaceTexture, and crop in OpenGL, if you want a live preview.