snachmsm
Reputation: 19273
MediaMuxer video file size reducing (re-compress, decrease resolution)
I'm looking for efficient way to reduce some video weight (as a File, for upload) and obvious answer for that is: lets reduce resolution! (fullHD or 4K not needed, simple HD is sufficient for me) I've tried lot of ways which should work through lot of APIs (needed 10) and best way was using android-ffmpeg-java, BUT... on my pretty fast almost-current flagship device whole process lasts about length_of_video*4 seconds and also this lib weight is 9 Mb, this amount increases my app size... No wai! (12 Mb to 1 Mb is nice result, but still too many flaws)
So I've decided to use native Android ways to do this, MediaMuxer and MediaCodec - they are available from API18 and API16 respectivelly (older devices users: sorry; but they also often have "lower-res" camera). Below method almost works - MediaMuxer do NOT respect MediaFormat.KEY_WIDTH and MediaFormat.KEY_HEIGHT - extracted File is "re-compressed", weight is a bit smaller, but resolution is the same as in original video File...
So, question: How to compress and re-scale/change resolution of video using MediaMuxer and other accompanying classes and methods?
public File getCompressedFile(String videoPath) throws IOException{
MediaExtractor extractor = new MediaExtractor();
extractor.setDataSource(videoPath);
int trackCount = extractor.getTrackCount();
String filePath = videoPath.substring(0, videoPath.lastIndexOf(File.separator));
String[] splitByDot = videoPath.split("\\.");
String ext="";
if(splitByDot!=null && splitByDot.length>1)
ext = splitByDot[splitByDot.length-1];
String fileName = videoPath.substring(videoPath.lastIndexOf(File.separator)+1,
videoPath.length());
if(ext.length()>0)
fileName=fileName.replace("."+ext, "_out."+ext);
else
fileName=fileName.concat("_out");
final File outFile = new File(filePath, fileName);
if(!outFile.exists())
outFile.createNewFile();
MediaMuxer muxer = new MediaMuxer(outFile.getAbsolutePath(),
MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
HashMap<Integer, Integer> indexMap = new HashMap<Integer, Integer>(trackCount);
for (int i = 0; i < trackCount; i++) {
extractor.selectTrack(i);
MediaFormat format = extractor.getTrackFormat(i);
String mime = format.getString(MediaFormat.KEY_MIME);
if(mime!=null && mime.startsWith("video")){
int currWidth = format.getInteger(MediaFormat.KEY_WIDTH);
int currHeight = format.getInteger(MediaFormat.KEY_HEIGHT);
format.setInteger(MediaFormat.KEY_WIDTH, currWidth>currHeight ? 960 : 540);
format.setInteger(MediaFormat.KEY_HEIGHT, currWidth>currHeight ? 540 : 960);
//API19 MediaFormat.KEY_MAX_WIDTH and KEY_MAX_HEIGHT
format.setInteger("max-width", format.getInteger(MediaFormat.KEY_WIDTH));
format.setInteger("max-height", format.getInteger(MediaFormat.KEY_HEIGHT));
}
int dstIndex = muxer.addTrack(format);
indexMap.put(i, dstIndex);
}
boolean sawEOS = false;
int bufferSize = 256 * 1024;
int offset = 100;
ByteBuffer dstBuf = ByteBuffer.allocate(bufferSize);
MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
muxer.start();
while (!sawEOS) {
bufferInfo.offset = offset;
bufferInfo.size = extractor.readSampleData(dstBuf, offset);
if (bufferInfo.size < 0) {
sawEOS = true;
bufferInfo.size = 0;
} else {
bufferInfo.presentationTimeUs = extractor.getSampleTime();
bufferInfo.flags = extractor.getSampleFlags();
int trackIndex = extractor.getSampleTrackIndex();
muxer.writeSampleData(indexMap.get(trackIndex), dstBuf,
bufferInfo);
extractor.advance();
}
}
muxer.stop();
muxer.release();
return outFile;
}
PS. lot of usefull stuff about muxer here, above code bases on MediaMuxerTest.java, method cloneMediaUsingMuxer
Upvotes: 9
Views: 12523
Answers (9)
Michael N
Reputation: 559
TextureRender.kt
import android.graphics.Bitmap
import android.graphics.SurfaceTexture
import android.opengl.GLES11Ext
import android.opengl.GLES20
import android.opengl.GLES30
import android.opengl.Matrix
import android.util.Log
import java.io.FileOutputStream
import java.io.IOException
import java.lang.RuntimeException
import java.nio.ByteBuffer
import java.nio.ByteOrder
import java.nio.FloatBuffer
/*
* Copyright (C) 2013 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
var gles = 2
/**
* Code for rendering a texture onto a surface using OpenGL ES 2.0.
*/
internal class TextureRender {
private val mTriangleVerticesData = floatArrayOf( // X, Y, Z, U, V
-1.0f, -1.0f, 0f, 0f, 0f,
1.0f, -1.0f, 0f, 1f, 0f,
-1.0f, 1.0f, 0f, 0f, 1f,
1.0f, 1.0f, 0f, 1f, 1f
)
private val mTriangleVertices: FloatBuffer
private val mMVPMatrix = FloatArray(16)
private val mSTMatrix = FloatArray(16)
private var mProgram = 0
var textureId = -12345
private set
private var muMVPMatrixHandle = 0
private var muSTMatrixHandle = 0
private var maPositionHandle = 0
private var maTextureHandle = 0
fun drawFrame(st: SurfaceTexture) {
if (gles == 2) {
checkGlError("onDrawFrame start")
st.getTransformMatrix(mSTMatrix)
GLES20.glClearColor(0.0f, 1.0f, 0.0f, 1.0f)
GLES20.glClear(GLES20.GL_DEPTH_BUFFER_BIT or GLES20.GL_COLOR_BUFFER_BIT)
GLES20.glUseProgram(mProgram)
checkGlError("glUseProgram")
GLES20.glActiveTexture(GLES20.GL_TEXTURE0)
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId)
mTriangleVertices.position(TRIANGLE_VERTICES_DATA_POS_OFFSET)
GLES20.glVertexAttribPointer(
maPositionHandle, 3, GLES20.GL_FLOAT, false,
TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices
)
checkGlError("glVertexAttribPointer maPosition")
GLES20.glEnableVertexAttribArray(maPositionHandle)
checkGlError("glEnableVertexAttribArray maPositionHandle")
mTriangleVertices.position(TRIANGLE_VERTICES_DATA_UV_OFFSET)
GLES20.glVertexAttribPointer(
maTextureHandle, 2, GLES20.GL_FLOAT, false,
TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices
)
checkGlError("glVertexAttribPointer maTextureHandle")
GLES20.glEnableVertexAttribArray(maTextureHandle)
checkGlError("glEnableVertexAttribArray maTextureHandle")
Matrix.setIdentityM(mMVPMatrix, 0)
GLES20.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0)
GLES20.glUniformMatrix4fv(muSTMatrixHandle, 1, false, mSTMatrix, 0)
GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, 0, 4)
checkGlError("glDrawArrays")
GLES20.glFinish()
} else {
checkGlError("onDrawFrame start")
st.getTransformMatrix(mSTMatrix)
GLES30.glClearColor(0.0f, 1.0f, 0.0f, 1.0f)
GLES30.glClear(GLES30.GL_DEPTH_BUFFER_BIT or GLES30.GL_COLOR_BUFFER_BIT)
GLES30.glUseProgram(mProgram)
checkGlError("glUseProgram")
GLES30.glActiveTexture(GLES30.GL_TEXTURE0)
GLES30.glBindTexture(GLES30.GL_TEXTURE_2D, textureId)
mTriangleVertices.position(TRIANGLE_VERTICES_DATA_POS_OFFSET)
GLES30.glVertexAttribPointer(
maPositionHandle, 3, GLES30.GL_FLOAT, false,
TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices
)
checkGlError("glVertexAttribPointer maPosition")
GLES30.glEnableVertexAttribArray(maPositionHandle)
checkGlError("glEnableVertexAttribArray maPositionHandle")
mTriangleVertices.position(TRIANGLE_VERTICES_DATA_UV_OFFSET)
GLES30.glVertexAttribPointer(
maTextureHandle, 2, GLES30.GL_FLOAT, false,
TRIANGLE_VERTICES_DATA_STRIDE_BYTES, mTriangleVertices
)
checkGlError("glVertexAttribPointer maTextureHandle")
GLES30.glEnableVertexAttribArray(maTextureHandle)
checkGlError("glEnableVertexAttribArray maTextureHandle")
Matrix.setIdentityM(mMVPMatrix, 0)
GLES30.glUniformMatrix4fv(muMVPMatrixHandle, 1, false, mMVPMatrix, 0)
GLES30.glUniformMatrix4fv(muSTMatrixHandle, 1, false, mSTMatrix, 0)
GLES30.glDrawArrays(GLES30.GL_TRIANGLE_STRIP, 0, 4)
checkGlError("glDrawArrays")
GLES30.glFinish()
}
}
/**
* Initializes GL state. Call this after the EGL surface has been created and made current.
*/
fun surfaceCreated() {
if (gles == 2) {
mProgram = createProgram(VERTEX_SHADER, FRAGMENT_SHADER)
if (mProgram == 0) {
throw RuntimeException("failed creating program")
}
maPositionHandle = GLES20.glGetAttribLocation(mProgram, "aPosition")
checkGlError("glGetAttribLocation aPosition")
if (maPositionHandle == -1) {
throw RuntimeException("Could not get attrib location for aPosition")
}
maTextureHandle = GLES20.glGetAttribLocation(mProgram, "aTextureCoord")
checkGlError("glGetAttribLocation aTextureCoord")
if (maTextureHandle == -1) {
throw RuntimeException("Could not get attrib location for aTextureCoord")
}
muMVPMatrixHandle = GLES20.glGetUniformLocation(mProgram, "uMVPMatrix")
checkGlError("glGetUniformLocation uMVPMatrix")
if (muMVPMatrixHandle == -1) {
throw RuntimeException("Could not get attrib location for uMVPMatrix")
}
muSTMatrixHandle = GLES20.glGetUniformLocation(mProgram, "uSTMatrix")
checkGlError("glGetUniformLocation uSTMatrix")
if (muSTMatrixHandle == -1) {
throw RuntimeException("Could not get attrib location for uSTMatrix")
}
val textures = IntArray(1)
GLES20.glGenTextures(1, textures, 0)
textureId = textures[0]
GLES20.glBindTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES, textureId)
checkGlError("glBindTexture mTextureID")
GLES20.glTexParameterf(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MIN_FILTER,
GLES20.GL_NEAREST.toFloat()
)
GLES20.glTexParameterf(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_MAG_FILTER,
GLES20.GL_LINEAR.toFloat()
)
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_S,
GLES20.GL_CLAMP_TO_EDGE
)
GLES20.glTexParameteri(
GLES11Ext.GL_TEXTURE_EXTERNAL_OES, GLES20.GL_TEXTURE_WRAP_T,
GLES20.GL_CLAMP_TO_EDGE
)
checkGlError("glTexParameter")
} else {
mProgram = createProgram(VERTEX_SHADER, FRAGMENT_SHADER)
if (mProgram == 0) {
throw RuntimeException("failed creating program")
}
maPositionHandle = GLES30.glGetAttribLocation(mProgram, "aPosition")
checkGlError("glGetAttribLocation aPosition")
if (maPositionHandle == -1) {
throw RuntimeException("Could not get attrib location for aPosition")
}
maTextureHandle = GLES30.glGetAttribLocation(mProgram, "aTextureCoord")
checkGlError("glGetAttribLocation aTextureCoord")
if (maTextureHandle == -1) {
throw RuntimeException("Could not get attrib location for aTextureCoord")
}
muMVPMatrixHandle = GLES30.glGetUniformLocation(mProgram, "uMVPMatrix")
checkGlError("glGetUniformLocation uMVPMatrix")
if (muMVPMatrixHandle == -1) {
throw RuntimeException("Could not get attrib location for uMVPMatrix")
}
muSTMatrixHandle = GLES30.glGetUniformLocation(mProgram, "uSTMatrix")
checkGlError("glGetUniformLocation uSTMatrix")
if (muSTMatrixHandle == -1) {
throw RuntimeException("Could not get attrib location for uSTMatrix")
}
val textures = IntArray(1)
GLES30.glGenTextures(1, textures, 0)
textureId = textures[0]
GLES30.glBindTexture(GLES30.GL_TEXTURE_2D, textureId)
checkGlError("glBindTexture mTextureID")
GLES30.glTexParameterf(
GLES30.GL_TEXTURE_2D, GLES30.GL_TEXTURE_MIN_FILTER,
GLES30.GL_NEAREST.toFloat()
)
GLES30.glTexParameterf(
GLES30.GL_TEXTURE_2D, GLES30.GL_TEXTURE_MAG_FILTER,
GLES30.GL_LINEAR.toFloat()
)
GLES30.glTexParameteri(
GLES30.GL_TEXTURE_2D, GLES30.GL_TEXTURE_WRAP_S,
GLES30.GL_CLAMP_TO_EDGE
)
GLES30.glTexParameteri(
GLES30.GL_TEXTURE_2D, GLES30.GL_TEXTURE_WRAP_T,
GLES30.GL_CLAMP_TO_EDGE
)
checkGlError("glTexParameter")
}
}
/**
* Replaces the fragment shader.
*/
fun changeFragmentShader(fragmentShader: String) {
if (gles == 2) {
GLES20.glDeleteProgram(mProgram)
mProgram = createProgram(VERTEX_SHADER, fragmentShader)
if (mProgram == 0) {
throw RuntimeException("failed creating program")
}
} else {
GLES30.glDeleteProgram(mProgram)
mProgram = createProgram(VERTEX_SHADER, fragmentShader)
if (mProgram == 0) {
throw RuntimeException("failed creating program")
}
}
}
private fun loadShader(shaderType: Int, source: String): Int {
if (gles == 2) {
var shader = GLES20.glCreateShader(shaderType)
checkGlError("glCreateShader type=$shaderType")
GLES20.glShaderSource(shader, source)
GLES20.glCompileShader(shader)
val compiled = IntArray(1)
GLES20.glGetShaderiv(shader, GLES20.GL_COMPILE_STATUS, compiled, 0)
if (compiled[0] == 0) {
Log.e(TAG, "Could not compile shader $shaderType:")
Log.e(TAG, " " + GLES20.glGetShaderInfoLog(shader))
GLES20.glDeleteShader(shader)
shader = 0
}
return shader
} else {
var shader = GLES30.glCreateShader(shaderType)
checkGlError("glCreateShader type=$shaderType")
GLES30.glShaderSource(shader, source)
GLES30.glCompileShader(shader)
val compiled = IntArray(1)
GLES30.glGetShaderiv(shader, GLES30.GL_COMPILE_STATUS, compiled, 0)
if (compiled[0] == 0) {
Log.e(TAG, "Could not compile shader $shaderType:")
Log.e(TAG, " " + GLES30.glGetShaderInfoLog(shader))
GLES30.glDeleteShader(shader)
shader = 0
}
return shader
}
}
private fun createProgram(vertexSource: String, fragmentSource: String): Int {
if (gles == 2) {
val vertexShader = loadShader(GLES20.GL_VERTEX_SHADER, vertexSource)
if (vertexShader == 0) {
return 0
}
val pixelShader = loadShader(GLES20.GL_FRAGMENT_SHADER, fragmentSource)
if (pixelShader == 0) {
return 0
}
var program = GLES20.glCreateProgram()
checkGlError("glCreateProgram")
if (program == 0) {
Log.e(TAG, "Could not create program")
}
GLES20.glAttachShader(program, vertexShader)
checkGlError("glAttachShader")
GLES20.glAttachShader(program, pixelShader)
checkGlError("glAttachShader")
GLES20.glLinkProgram(program)
val linkStatus = IntArray(1)
GLES20.glGetProgramiv(program, GLES20.GL_LINK_STATUS, linkStatus, 0)
if (linkStatus[0] != GLES20.GL_TRUE) {
Log.e(TAG, "Could not link program: ")
Log.e(TAG, GLES20.glGetProgramInfoLog(program))
GLES20.glDeleteProgram(program)
program = 0
}
return program
} else {
val vertexShader = loadShader(GLES30.GL_VERTEX_SHADER, vertexSource)
if (vertexShader == 0) {
return 0
}
val pixelShader = loadShader(GLES30.GL_FRAGMENT_SHADER, fragmentSource)
if (pixelShader == 0) {
return 0
}
var program = GLES30.glCreateProgram()
checkGlError("glCreateProgram")
if (program == 0) {
Log.e(TAG, "Could not create program")
}
GLES30.glAttachShader(program, vertexShader)
checkGlError("glAttachShader")
GLES30.glAttachShader(program, pixelShader)
checkGlError("glAttachShader")
GLES30.glLinkProgram(program)
val linkStatus = IntArray(1)
GLES30.glGetProgramiv(program, GLES30.GL_LINK_STATUS, linkStatus, 0)
if (linkStatus[0] != GLES30.GL_TRUE) {
Log.e(TAG, "Could not link program: ")
Log.e(TAG, GLES30.glGetProgramInfoLog(program))
GLES30.glDeleteProgram(program)
program = 0
}
return program
}
}
fun checkGlError(op: String) {
if (gles == 2) {
var error: Int
while (GLES20.glGetError().also { error = it } != GLES20.GL_NO_ERROR) {
Log.e(TAG, "$op: glError $error")
throw RuntimeException("$op: glError $error")
}
} else {
var error: Int
while (GLES30.glGetError().also { error = it } != GLES30.GL_NO_ERROR) {
Log.e(TAG, "$op: glError $error")
throw RuntimeException("$op: glError $error")
}
}
}
companion object {
private const val TAG = "TextureRender"
private const val FLOAT_SIZE_BYTES = 4
private const val TRIANGLE_VERTICES_DATA_STRIDE_BYTES = 5 * FLOAT_SIZE_BYTES
private const val TRIANGLE_VERTICES_DATA_POS_OFFSET = 0
private const val TRIANGLE_VERTICES_DATA_UV_OFFSET = 3
private const val VERTEX_SHADER = "uniform mat4 uMVPMatrix;\n" +
"uniform mat4 uSTMatrix;\n" +
"attribute vec4 aPosition;\n" +
"attribute vec4 aTextureCoord;\n" +
"varying vec2 vTextureCoord;\n" +
"void main() {\n" +
" gl_Position = uMVPMatrix * aPosition;\n" +
" vTextureCoord = (uSTMatrix * aTextureCoord).xy;\n" +
"}\n"
private const val FRAGMENT_SHADER = "#extension GL_OES_EGL_image_external : require\n" +
"precision mediump float;\n" + // highp here doesn't seem to matter
"varying vec2 vTextureCoord;\n" +
"uniform samplerExternalOES sTexture;\n" +
"void main() {\n" +
" gl_FragColor = texture2D(sTexture, vTextureCoord);\n" +
"}\n"
}
init {
mTriangleVertices = ByteBuffer.allocateDirect(
mTriangleVerticesData.size * FLOAT_SIZE_BYTES
)
.order(ByteOrder.nativeOrder()).asFloatBuffer()
mTriangleVertices.put(mTriangleVerticesData).position(0)
Matrix.setIdentityM(mSTMatrix, 0)
}
}
Upvotes: 0
Michael N
Reputation: 559
InputSurface.kt
/*
* Copyright (C) 2013 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Holds state associated with a Surface used for MediaCodec encoder input.
*
*
* The constructor takes a Surface obtained from MediaCodec.createInputSurface(), and uses that
* to create an EGL window surface. Calls to eglSwapBuffers() cause a frame of data to be sent
* to the video encoder.
*/
internal class InputSurface(surface: Surface?) {
private var mEGLDisplay = EGL14.EGL_NO_DISPLAY
private var mEGLContext = EGL14.EGL_NO_CONTEXT
private var mEGLSurface = EGL14.EGL_NO_SURFACE
/**
* Returns the Surface that the MediaCodec receives buffers from.
*/
var surface: Surface?
private set
/**
* Prepares EGL. We want a GLES 2.0 context and a surface that supports recording.
*/
private fun eglSetup() {
mEGLDisplay = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
throw RuntimeException("unable to get EGL14 display")
}
val version = IntArray(2)
if (!EGL14.eglInitialize(mEGLDisplay, version, 0, version, 1)) {
mEGLDisplay = null
throw RuntimeException("unable to initialize EGL14")
}
// Configure EGL for recordable and OpenGL ES 2.0. We want enough RGB bits
// to minimize artifacts from possible YUV conversion.
val attribList = intArrayOf(
EGL14.EGL_RED_SIZE, 8,
EGL14.EGL_GREEN_SIZE, 8,
EGL14.EGL_BLUE_SIZE, 8,
EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT,
EGL_RECORDABLE_ANDROID, 1,
EGL14.EGL_NONE
)
val configs = arrayOfNulls<EGLConfig>(1)
val numConfigs = IntArray(1)
if (!EGL14.eglChooseConfig(
mEGLDisplay, attribList, 0, configs, 0, configs.size,
numConfigs, 0
)
) {
throw RuntimeException("unable to find RGB888+recordable ES2 EGL config")
}
// Configure context for OpenGL ES 2.0.
val attrib_list = intArrayOf(
EGL14.EGL_CONTEXT_CLIENT_VERSION, 2,
EGL14.EGL_NONE
)
mEGLContext = EGL14.eglCreateContext(
mEGLDisplay, configs[0], EGL14.EGL_NO_CONTEXT,
attrib_list, 0
)
checkEglError("eglCreateContext")
if (mEGLContext == null) {
throw RuntimeException("null context")
}
// Create a window surface, and attach it to the Surface we received.
val surfaceAttribs = intArrayOf(
EGL14.EGL_NONE
)
mEGLSurface = EGL14.eglCreateWindowSurface(
mEGLDisplay, configs[0], surface,
surfaceAttribs, 0
)
checkEglError("eglCreateWindowSurface")
if (mEGLSurface == null) {
throw RuntimeException("surface was null")
}
}
/**
* Discard all resources held by this class, notably the EGL context. Also releases the
* Surface that was passed to our constructor.
*/
fun release() {
if (mEGLDisplay !== EGL14.EGL_NO_DISPLAY) {
EGL14.eglDestroySurface(mEGLDisplay, mEGLSurface)
EGL14.eglDestroyContext(mEGLDisplay, mEGLContext)
EGL14.eglReleaseThread()
EGL14.eglTerminate(mEGLDisplay)
}
surface!!.release()
mEGLDisplay = EGL14.EGL_NO_DISPLAY
mEGLContext = EGL14.EGL_NO_CONTEXT
mEGLSurface = EGL14.EGL_NO_SURFACE
surface = null
}
/**
* Makes our EGL context and surface current.
*/
fun makeCurrent() {
if (!EGL14.eglMakeCurrent(mEGLDisplay, mEGLSurface, mEGLSurface, mEGLContext)) {
throw RuntimeException("eglMakeCurrent failed")
}
}
fun makeUnCurrent() {
if (!EGL14.eglMakeCurrent(
mEGLDisplay, EGL14.EGL_NO_SURFACE, EGL14.EGL_NO_SURFACE,
EGL14.EGL_NO_CONTEXT
)
) {
throw RuntimeException("eglMakeCurrent failed")
}
}
/**
* Calls eglSwapBuffers. Use this to "publish" the current frame.
*/
fun swapBuffers(): Boolean {
//println("swapBuffers")
return EGL14.eglSwapBuffers(mEGLDisplay, mEGLSurface)
}
/**
* Queries the surface's width.
*/
val width: Int
get() {
val value = IntArray(1)
EGL14.eglQuerySurface(mEGLDisplay, mEGLSurface, EGL14.EGL_WIDTH, value, 0)
return value[0]
}
/**
* Queries the surface's height.
*/
val height: Int
get() {
val value = IntArray(1)
EGL14.eglQuerySurface(mEGLDisplay, mEGLSurface, EGL14.EGL_HEIGHT, value, 0)
return value[0]
}
/**
* Sends the presentation time stamp to EGL. Time is expressed in nanoseconds.
*/
fun setPresentationTime(nsecs: Long) {
EGLExt.eglPresentationTimeANDROID(mEGLDisplay, mEGLSurface, nsecs)
}
/**
* Checks for EGL errors.
*/
private fun checkEglError(msg: String) {
var error: Int
if (EGL14.eglGetError().also { error = it } != EGL14.EGL_SUCCESS) {
throw RuntimeException(msg + ": EGL error: 0x" + Integer.toHexString(error))
}
}
companion object {
private const val TAG = "InputSurface"
private const val VERBOSE = false
private const val EGL_RECORDABLE_ANDROID = 0x3142
}
/**
* Creates an InputSurface from a Surface.
*/
init {
if (surface == null) {
throw NullPointerException()
}
this.surface = surface
eglSetup()
}
}
Upvotes: 0
Michael N
Reputation: 559
OutputSurface.kt
/*
* Copyright (C) 2013 The Android Open Source Project
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
/**
* Holds state associated with a Surface used for MediaCodec decoder output.
*
*
* The (width,height) constructor for this class will prepare GL, create a SurfaceTexture,
* and then create a Surface for that SurfaceTexture. The Surface can be passed to
* MediaCodec.configure() to receive decoder output. When a frame arrives, we latch the
* texture with updateTexImage, then render the texture with GL to a pbuffer.
*
*
* The no-arg constructor skips the GL preparation step and doesn't allocate a pbuffer.
* Instead, it just creates the Surface and SurfaceTexture, and when a frame arrives
* we just draw it on whatever surface is current.
*
*
* By default, the Surface will be using a BufferQueue in asynchronous mode, so we
* can potentially drop frames.
*/
internal class OutputSurface : OnFrameAvailableListener {
private var mEGLDisplay = EGL14.EGL_NO_DISPLAY
private var mEGLContext = EGL14.EGL_NO_CONTEXT
private var mEGLSurface = EGL14.EGL_NO_SURFACE
private var mSurfaceTexture: SurfaceTexture? = null
/**
* Returns the Surface that we draw onto.
*/
var surface: Surface? = null
private set
private val mFrameSyncObject = Object() // guards mFrameAvailable
private var mFrameAvailable = false
private var mTextureRender: TextureRender? = null
/**
* Creates an OutputSurface backed by a pbuffer with the specifed dimensions. The new
* EGL context and surface will be made current. Creates a Surface that can be passed
* to MediaCodec.configure().
*/
constructor(width: Int, height: Int) {
println("OutputSurface constructor width: $width height: $height")
require(!(width <= 0 || height <= 0))
eglSetup(width, height)
makeCurrent()
setup()
}
/**
* Creates an OutputSurface using the current EGL context (rather than establishing a
* new one). Creates a Surface that can be passed to MediaCodec.configure().
*/
constructor() {
println("OutputSurface constructor")
setup()
}
/**
* Creates instances of TextureRender and SurfaceTexture, and a Surface associated
* with the SurfaceTexture.
*/
private fun setup() {
println("OutputSurface setup")
mTextureRender = TextureRender()
mTextureRender!!.surfaceCreated()
// Even if we don't access the SurfaceTexture after the constructor returns, we
// still need to keep a reference to it. The Surface doesn't retain a reference
// at the Java level, so if we don't either then the object can get GCed, which
// causes the native finalizer to run.
if (VERBOSE) Log.d(TAG, "textureID=" + mTextureRender!!.textureId)
mSurfaceTexture = SurfaceTexture(mTextureRender!!.textureId)
// This doesn't work if OutputSurface is created on the thread that CTS started for
// these test cases.
//
// The CTS-created thread has a Looper, and the SurfaceTexture constructor will
// create a Handler that uses it. The "frame available" message is delivered
// there, but since we're not a Looper-based thread we'll never see it. For
// this to do anything useful, OutputSurface must be created on a thread without
// a Looper, so that SurfaceTexture uses the main application Looper instead.
//
// Java language note: passing "this" out of a constructor is generally unwise,
// but we should be able to get away with it here.
mSurfaceTexture!!.setOnFrameAvailableListener(this)
surface = Surface(mSurfaceTexture)
}
/**
* Prepares EGL. We want a GLES 2.0 context and a surface that supports pbuffer.
*/
private fun eglSetup(width: Int, height: Int) {
mEGLDisplay = EGL14.eglGetDisplay(EGL14.EGL_DEFAULT_DISPLAY)
if (mEGLDisplay === EGL14.EGL_NO_DISPLAY) {
throw RuntimeException("unable to get EGL14 display")
}
val version = IntArray(2)
if (!EGL14.eglInitialize(mEGLDisplay, version, 0, version, 1)) {
mEGLDisplay = null
throw RuntimeException("unable to initialize EGL14")
}
// Configure EGL for pbuffer and OpenGL ES 2.0. We want enough RGB bits
// to be able to tell if the frame is reasonable.
val attribList = intArrayOf(
EGL14.EGL_RED_SIZE, 8,
EGL14.EGL_GREEN_SIZE, 8,
EGL14.EGL_BLUE_SIZE, 8,
EGL14.EGL_RENDERABLE_TYPE, EGL14.EGL_OPENGL_ES2_BIT,
EGL14.EGL_SURFACE_TYPE, EGL14.EGL_PBUFFER_BIT,
EGL14.EGL_NONE
)
val configs = arrayOfNulls<EGLConfig>(1)
val numConfigs = IntArray(1)
if (!EGL14.eglChooseConfig(
mEGLDisplay, attribList, 0, configs, 0, configs.size,
numConfigs, 0
)
) {
throw RuntimeException("unable to find RGB888+recordable ES2 EGL config")
}
// Configure context for OpenGL ES 2.0.
val attrib_list = intArrayOf(
EGL14.EGL_CONTEXT_CLIENT_VERSION, 2,
EGL14.EGL_NONE
)
mEGLContext = EGL14.eglCreateContext(
mEGLDisplay, configs[0], EGL14.EGL_NO_CONTEXT,
attrib_list, 0
)
checkEglError("eglCreateContext")
if (mEGLContext == null) {
throw RuntimeException("null context")
}
// Create a pbuffer surface. By using this for output, we can use glReadPixels
// to test values in the output.
val surfaceAttribs = intArrayOf(
EGL14.EGL_WIDTH, width,
EGL14.EGL_HEIGHT, height,
EGL14.EGL_NONE
)
mEGLSurface = EGL14.eglCreatePbufferSurface(mEGLDisplay, configs[0], surfaceAttribs, 0)
checkEglError("eglCreatePbufferSurface")
if (mEGLSurface == null) {
throw RuntimeException("surface was null")
}
}
/**
* Discard all resources held by this class, notably the EGL context.
*/
fun release() {
if (mEGLDisplay !== EGL14.EGL_NO_DISPLAY) {
EGL14.eglDestroySurface(mEGLDisplay, mEGLSurface)
EGL14.eglDestroyContext(mEGLDisplay, mEGLContext)
EGL14.eglReleaseThread()
EGL14.eglTerminate(mEGLDisplay)
}
surface!!.release()
// this causes a bunch of warnings that appear harmless but might confuse someone:
// W BufferQueue: [unnamed-3997-2] cancelBuffer: BufferQueue has been abandoned!
//mSurfaceTexture.release();
mEGLDisplay = EGL14.EGL_NO_DISPLAY
mEGLContext = EGL14.EGL_NO_CONTEXT
mEGLSurface = EGL14.EGL_NO_SURFACE
mTextureRender = null
surface = null
mSurfaceTexture = null
}
/**
* Makes our EGL context and surface current.
*/
private fun makeCurrent() {
if (!EGL14.eglMakeCurrent(mEGLDisplay, mEGLSurface, mEGLSurface, mEGLContext)) {
throw RuntimeException("eglMakeCurrent failed")
}
}
/**
* Replaces the fragment shader.
*/
fun changeFragmentShader(fragmentShader: String?) {
if (fragmentShader != null) {
mTextureRender?.changeFragmentShader(fragmentShader)
}
}
/**
* Latches the next buffer into the texture. Must be called from the thread that created
* the OutputSurface object, after the onFrameAvailable callback has signaled that new
* data is available.
*/
fun awaitNewImage() {
//println("awaitNewImage")
val TIMEOUT_MS = 500
synchronized(mFrameSyncObject) {
while (!mFrameAvailable) {
try {
// Wait for onFrameAvailable() to signal us. Use a timeout to avoid
// stalling the test if it doesn't arrive.
mFrameSyncObject.wait(TIMEOUT_MS.toLong())
if (!mFrameAvailable) {
// TODO: if "spurious wakeup", continue while loop
//throw RuntimeException("Surface frame wait timed out")
}
} catch (ie: InterruptedException) {
// shouldn't happen
throw RuntimeException(ie)
}
}
mFrameAvailable = false
}
// Latch the data.
mTextureRender?.checkGlError("before updateTexImage")
mSurfaceTexture!!.updateTexImage()
}
/**
* Draws the data from SurfaceTexture onto the current EGL surface.
*/
fun drawImage() {
mSurfaceTexture?.let { mTextureRender?.drawFrame(it) }
}
override fun onFrameAvailable(st: SurfaceTexture) {
//println("onFrameAvailable")
if (VERBOSE) Log.d(TAG, "new frame available")
synchronized(mFrameSyncObject) {
if (mFrameAvailable) {
throw RuntimeException("mFrameAvailable already set, frame could be dropped")
}
mFrameAvailable = true
mFrameSyncObject.notifyAll()
}
}
/**
* Checks for EGL errors.
*/
private fun checkEglError(msg: String) {
var error: Int
if (EGL14.eglGetError().also { error = it } != EGL14.EGL_SUCCESS) {
throw RuntimeException(msg + ": EGL error: 0x" + Integer.toHexString(error))
}
}
companion object {
private const val TAG = "OutputSurface"
private const val VERBOSE = false
}
}
Upvotes: 0
Michael N
Reputation: 559
VideoResolutionChanger.kt
class VideoResolutionChanger {
private val TIMEOUT_USEC = 10000
private val OUTPUT_VIDEO_MIME_TYPE = "video/avc"
private val OUTPUT_VIDEO_BIT_RATE = 2048 * 1024
private val OUTPUT_VIDEO_FRAME_RATE = 60
private val OUTPUT_VIDEO_IFRAME_INTERVAL = 1
private val OUTPUT_VIDEO_COLOR_FORMAT = MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface
private val OUTPUT_AUDIO_MIME_TYPE = "audio/mp4a-latm"
private val OUTPUT_AUDIO_CHANNEL_COUNT = 2
private val OUTPUT_AUDIO_BIT_RATE = 128 * 1024
private val OUTPUT_AUDIO_AAC_PROFILE = MediaCodecInfo.CodecProfileLevel.AACObjectHE
private val OUTPUT_AUDIO_SAMPLE_RATE_HZ = 44100
private var mWidth = 1920
private var mHeight = 1080
private var mOutputFile : String? = null
private var mInputFile : String? = null
private var mTotalTime : Int = 0
@Throws(Throwable::class)
fun changeResolution(f: File): String? {
mInputFile = f.absolutePath
val filePath : String? = mInputFile!!.substring(0, mInputFile!!.lastIndexOf(File.separator))
val splitByDot: Array<String> = mInputFile!!.split("\\.").toTypedArray()
var ext = ""
if (splitByDot.size > 1) ext = splitByDot[splitByDot.size - 1]
var fileName: String = mInputFile!!.substring(
mInputFile!!.lastIndexOf(File.separator) + 1,
mInputFile!!.length
)
fileName = if (ext.length > 0) fileName.replace(".$ext", "_out.mp4") else fileName + "_out.mp4"
mOutputFile = outFile.getAbsolutePath()
ChangerWrapper.changeResolutionInSeparatedThread(this)
return mOutputFile
}
private class ChangerWrapper private constructor(private val mChanger: VideoResolutionChanger) :
Runnable {
private var mThrowable : Throwable? = null
override fun run() {
try {
mChanger.prepareAndChangeResolution()
} catch (th: Throwable) {
mThrowable = th
}
}
companion object {
@Throws(Throwable::class)
fun changeResolutionInSeparatedThread(changer: VideoResolutionChanger) {
val wrapper = ChangerWrapper(changer)
val th = Thread(wrapper, ChangerWrapper::class.java.simpleName)
th.start()
th.join()
if (wrapper.mThrowable != null) throw wrapper.mThrowable!!
}
}
}
@Throws(Exception::class)
private fun prepareAndChangeResolution() {
var exception: Exception? = null
val videoCodecInfo = selectCodec(OUTPUT_VIDEO_MIME_TYPE) ?: return
val audioCodecInfo = selectCodec(OUTPUT_AUDIO_MIME_TYPE) ?: return
var videoExtractor : MediaExtractor? = null
var audioExtractor : MediaExtractor? = null
var outputSurface : OutputSurface? = null
var videoDecoder : MediaCodec? = null
var audioDecoder : MediaCodec? = null
var videoEncoder : MediaCodec? = null
var audioEncoder : MediaCodec? = null
var muxer : MediaMuxer? = null
var inputSurface : InputSurface? = null
try {
videoExtractor = createExtractor()
val videoInputTrack = getAndSelectVideoTrackIndex(videoExtractor)
val inputFormat = videoExtractor!!.getTrackFormat(videoInputTrack)
val m = MediaMetadataRetriever()
m.setDataSource(mInputFile)
var inputWidth: Int
var inputHeight: Int
try {
inputWidth =
m.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_WIDTH)!!.toInt()
inputHeight =
m.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT)!!.toInt()
mTotalTime =
m.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION)!!.toInt() * 1000
} catch (e: Exception) {
val thumbnail = m.frameAtTime
inputWidth = thumbnail!!.width
inputHeight = thumbnail.height
thumbnail.recycle()
}
if (inputWidth > inputHeight) {
if (mWidth < mHeight) {
val w = mWidth
mWidth = mHeight
mHeight = w
}
} else {
if (mWidth > mHeight) {
val w = mWidth
mWidth = mHeight
mHeight = w
}
}
val outputVideoFormat =
MediaFormat.createVideoFormat(OUTPUT_VIDEO_MIME_TYPE, mWidth, mHeight)
outputVideoFormat.setInteger(
MediaFormat.KEY_COLOR_FORMAT, OUTPUT_VIDEO_COLOR_FORMAT
)
outputVideoFormat.setInteger(MediaFormat.KEY_BIT_RATE, OUTPUT_VIDEO_BIT_RATE)
outputVideoFormat.setInteger(MediaFormat.KEY_FRAME_RATE, OUTPUT_VIDEO_FRAME_RATE)
outputVideoFormat.setInteger(
MediaFormat.KEY_I_FRAME_INTERVAL, OUTPUT_VIDEO_IFRAME_INTERVAL
)
val inputSurfaceReference: AtomicReference<Surface> = AtomicReference<Surface>()
videoEncoder = createVideoEncoder(
videoCodecInfo, outputVideoFormat, inputSurfaceReference
)
inputSurface = InputSurface(inputSurfaceReference.get())
inputSurface.makeCurrent()
outputSurface = OutputSurface()
videoDecoder = createVideoDecoder(inputFormat, outputSurface!!.surface!!);
audioExtractor = createExtractor()
val audioInputTrack = getAndSelectAudioTrackIndex(audioExtractor)
val inputAudioFormat = audioExtractor!!.getTrackFormat(audioInputTrack)
val outputAudioFormat = MediaFormat.createAudioFormat(
inputAudioFormat.getString(MediaFormat.KEY_MIME)!!,
inputAudioFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE),
inputAudioFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT)
)
outputAudioFormat.setInteger(MediaFormat.KEY_BIT_RATE, OUTPUT_AUDIO_BIT_RATE)
outputAudioFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, OUTPUT_AUDIO_AAC_PROFILE)
audioEncoder = createAudioEncoder(audioCodecInfo, outputAudioFormat)
audioDecoder = createAudioDecoder(inputAudioFormat)
muxer = MediaMuxer(mOutputFile!!, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4)
changeResolution(
videoExtractor, audioExtractor,
videoDecoder, videoEncoder,
audioDecoder, audioEncoder,
muxer, inputSurface, outputSurface
)
} finally {
try {
videoExtractor?.release()
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
audioExtractor?.release()
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
if (videoDecoder != null) {
videoDecoder.stop()
videoDecoder.release()
}
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
outputSurface?.release()
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
if (videoEncoder != null) {
videoEncoder.stop()
videoEncoder.release()
}
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
if (audioDecoder != null) {
audioDecoder.stop()
audioDecoder.release()
}
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
if (audioEncoder != null) {
audioEncoder.stop()
audioEncoder.release()
}
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
if (muxer != null) {
muxer.stop()
muxer.release()
}
} catch (e: Exception) {
if (exception == null) exception = e
}
try {
inputSurface?.release()
} catch (e: Exception) {
if (exception == null) exception = e
}
}
if (exception != null) throw exception
}
@Throws(IOException::class)
private fun createExtractor(): MediaExtractor? {
val extractor : MediaExtractor = MediaExtractor()
mInputFile?.let { extractor.setDataSource(it) }
return extractor
}
@Throws(IOException::class)
private fun createVideoDecoder(inputFormat: MediaFormat, surface: Surface): MediaCodec? {
val decoder = MediaCodec.createDecoderByType(getMimeTypeFor(inputFormat)!!)
decoder.configure(inputFormat, surface, null, 0)
decoder.start()
return decoder
}
@Throws(IOException::class)
private fun createVideoEncoder(
codecInfo: MediaCodecInfo, format: MediaFormat,
surfaceReference: AtomicReference<Surface>
): MediaCodec? {
val encoder = MediaCodec.createByCodecName(codecInfo.name)
encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
surfaceReference.set(encoder.createInputSurface())
encoder.start()
return encoder
}
@Throws(IOException::class)
private fun createAudioDecoder(inputFormat: MediaFormat): MediaCodec? {
val decoder = MediaCodec.createDecoderByType(getMimeTypeFor(inputFormat)!!)
decoder.configure(inputFormat, null, null, 0)
decoder.start()
return decoder
}
@Throws(IOException::class)
private fun createAudioEncoder(codecInfo: MediaCodecInfo, format: MediaFormat): MediaCodec? {
val encoder = MediaCodec.createByCodecName(codecInfo.name)
encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE)
encoder.start()
return encoder
}
private fun getAndSelectVideoTrackIndex(extractor: MediaExtractor?): Int {
for (index in 0 until extractor!!.trackCount) {
if (isVideoFormat(extractor.getTrackFormat(index))) {
extractor.selectTrack(index)
return index
}
}
return -1
}
private fun getAndSelectAudioTrackIndex(extractor: MediaExtractor?): Int {
for (index in 0 until extractor!!.trackCount) {
if (isAudioFormat(extractor.getTrackFormat(index))) {
extractor.selectTrack(index)
return index
}
}
return -1
}
private fun changeResolution(
videoExtractor: MediaExtractor?, audioExtractor: MediaExtractor?,
videoDecoder: MediaCodec?, videoEncoder: MediaCodec?,
audioDecoder: MediaCodec?, audioEncoder: MediaCodec?,
muxer: MediaMuxer,
inputSurface: InputSurface?, outputSurface: OutputSurface?
) {
var videoDecoderInputBuffers : Array<ByteBuffer?>? = null
var videoDecoderOutputBuffers : Array<ByteBuffer?>? = null
var videoEncoderOutputBuffers : Array<ByteBuffer?>? = null
var videoDecoderOutputBufferInfo : MediaCodec.BufferInfo? = null
var videoEncoderOutputBufferInfo : MediaCodec.BufferInfo? = null
videoDecoderInputBuffers = videoDecoder!!.inputBuffers
videoDecoderOutputBuffers = videoDecoder.outputBuffers
videoEncoderOutputBuffers = videoEncoder!!.outputBuffers
videoDecoderOutputBufferInfo = MediaCodec.BufferInfo()
videoEncoderOutputBufferInfo = MediaCodec.BufferInfo()
var audioDecoderInputBuffers : Array<ByteBuffer?>? = null
var audioDecoderOutputBuffers : Array<ByteBuffer>? = null
var audioEncoderInputBuffers : Array<ByteBuffer>? = null
var audioEncoderOutputBuffers : Array<ByteBuffer?>? = null
var audioDecoderOutputBufferInfo : MediaCodec.BufferInfo? = null
var audioEncoderOutputBufferInfo : MediaCodec.BufferInfo? = null
audioDecoderInputBuffers = audioDecoder!!.inputBuffers
audioDecoderOutputBuffers = audioDecoder.outputBuffers
audioEncoderInputBuffers = audioEncoder!!.inputBuffers
audioEncoderOutputBuffers = audioEncoder.outputBuffers
audioDecoderOutputBufferInfo = MediaCodec.BufferInfo()
audioEncoderOutputBufferInfo = MediaCodec.BufferInfo()
var encoderOutputVideoFormat : MediaFormat? = null
var encoderOutputAudioFormat : MediaFormat? = null
var outputVideoTrack = -1
var outputAudioTrack = -1
var videoExtractorDone = false
var videoDecoderDone = false
var videoEncoderDone = false
var audioExtractorDone = false
var audioDecoderDone = false
var audioEncoderDone = false
var pendingAudioDecoderOutputBufferIndex = -1
var muxing = false
while (!videoEncoderDone || !audioEncoderDone) {
while (!videoExtractorDone
&& (encoderOutputVideoFormat == null || muxing)
) {
val decoderInputBufferIndex = videoDecoder.dequeueInputBuffer(TIMEOUT_USEC.toLong())
if (decoderInputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
break
}
val decoderInputBuffer: ByteBuffer? =
videoDecoderInputBuffers[decoderInputBufferIndex]
val size = decoderInputBuffer?.let { videoExtractor!!.readSampleData(it, 0) }
val presentationTime = videoExtractor?.sampleTime
if (presentationTime != null) {
if (size != null) {
if (size >= 0) {
if (videoExtractor != null) {
videoDecoder.queueInputBuffer(
decoderInputBufferIndex,
0,
size,
presentationTime,
videoExtractor.sampleFlags
)
}
}
}
}
if (videoExtractor != null) {
videoExtractorDone = (!videoExtractor.advance() && size == -1)
}
if (videoExtractorDone) {
videoDecoder.queueInputBuffer(
decoderInputBufferIndex,
0,
0,
0,
MediaCodec.BUFFER_FLAG_END_OF_STREAM
)
}
break
}
while (!audioExtractorDone
&& (encoderOutputAudioFormat == null || muxing)
) {
val decoderInputBufferIndex = audioDecoder.dequeueInputBuffer(TIMEOUT_USEC.toLong())
if (decoderInputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
break
}
val decoderInputBuffer: ByteBuffer? =
audioDecoderInputBuffers[decoderInputBufferIndex]
val size = decoderInputBuffer?.let { audioExtractor!!.readSampleData(it, 0) }
val presentationTime = audioExtractor?.sampleTime
if (presentationTime != null) {
if (size != null) {
if (size >= 0) {
audioDecoder.queueInputBuffer(
decoderInputBufferIndex,
0,
size,
presentationTime,
audioExtractor.sampleFlags
)
}
}
}
if (audioExtractor != null) {
audioExtractorDone = (!audioExtractor.advance() && size == -1)
}
if (audioExtractorDone) {
audioDecoder.queueInputBuffer(
decoderInputBufferIndex,
0,
0,
0,
MediaCodec.BUFFER_FLAG_END_OF_STREAM
)
}
break
}
while (!videoDecoderDone
&& (encoderOutputVideoFormat == null || muxing)
) {
val decoderOutputBufferIndex = videoDecoder.dequeueOutputBuffer(
videoDecoderOutputBufferInfo, TIMEOUT_USEC.toLong()
)
if (decoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
break
}
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
videoDecoderOutputBuffers = videoDecoder.outputBuffers
break
}
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
decoderOutputVideoFormat = videoDecoder.outputFormat
break
}
val decoderOutputBuffer: ByteBuffer? =
videoDecoderOutputBuffers!![decoderOutputBufferIndex]
if (videoDecoderOutputBufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG
!= 0
) {
videoDecoder.releaseOutputBuffer(decoderOutputBufferIndex, false)
break
}
val render = videoDecoderOutputBufferInfo.size != 0
videoDecoder.releaseOutputBuffer(decoderOutputBufferIndex, render)
if (render) {
if (outputSurface != null) {
outputSurface.awaitNewImage()
outputSurface.drawImage()
}
if (inputSurface != null) {
inputSurface.setPresentationTime(
videoDecoderOutputBufferInfo.presentationTimeUs * 1000
)
inputSurface.swapBuffers()
}
}
if ((videoDecoderOutputBufferInfo.flags
and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0
) {
videoDecoderDone = true
videoEncoder.signalEndOfInputStream()
}
break
}
while (!audioDecoderDone && pendingAudioDecoderOutputBufferIndex == -1 && (encoderOutputAudioFormat == null || muxing)) {
val decoderOutputBufferIndex = audioDecoder.dequeueOutputBuffer(
audioDecoderOutputBufferInfo, TIMEOUT_USEC.toLong()
)
if (decoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
break
}
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
audioDecoderOutputBuffers = audioDecoder.outputBuffers
break
}
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
decoderOutputAudioFormat = audioDecoder.outputFormat
break
}
val decoderOutputBuffer: ByteBuffer =
audioDecoderOutputBuffers!![decoderOutputBufferIndex]
if (audioDecoderOutputBufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG
!= 0
) {
audioDecoder.releaseOutputBuffer(decoderOutputBufferIndex, false)
break
}
pendingAudioDecoderOutputBufferIndex = decoderOutputBufferIndex
break
}
while (pendingAudioDecoderOutputBufferIndex != -1) {
val encoderInputBufferIndex = audioEncoder.dequeueInputBuffer(TIMEOUT_USEC.toLong())
val encoderInputBuffer: ByteBuffer =
audioEncoderInputBuffers[encoderInputBufferIndex]
val size = audioDecoderOutputBufferInfo.size
val presentationTime = audioDecoderOutputBufferInfo.presentationTimeUs
if (size >= 0) {
val decoderOutputBuffer: ByteBuffer =
audioDecoderOutputBuffers!![pendingAudioDecoderOutputBufferIndex]
.duplicate()
decoderOutputBuffer.position(audioDecoderOutputBufferInfo.offset)
decoderOutputBuffer.limit(audioDecoderOutputBufferInfo.offset + size)
encoderInputBuffer.position(0)
encoderInputBuffer.put(decoderOutputBuffer)
audioEncoder.queueInputBuffer(
encoderInputBufferIndex,
0,
size,
presentationTime,
audioDecoderOutputBufferInfo.flags
)
}
audioDecoder.releaseOutputBuffer(pendingAudioDecoderOutputBufferIndex, false)
pendingAudioDecoderOutputBufferIndex = -1
if ((audioDecoderOutputBufferInfo.flags
and MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0
) audioDecoderDone = true
break
}
while (!videoEncoderDone
&& (encoderOutputVideoFormat == null || muxing)
) {
val encoderOutputBufferIndex = videoEncoder.dequeueOutputBuffer(
videoEncoderOutputBufferInfo, TIMEOUT_USEC.toLong()
)
if (encoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) break
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
videoEncoderOutputBuffers = videoEncoder.outputBuffers
break
}
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
encoderOutputVideoFormat = videoEncoder.outputFormat
break
}
val encoderOutputBuffer: ByteBuffer? =
videoEncoderOutputBuffers!![encoderOutputBufferIndex]
if (videoEncoderOutputBufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG
!= 0
) {
videoEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false)
break
}
if (videoEncoderOutputBufferInfo.size != 0) {
if (encoderOutputBuffer != null) {
muxer.writeSampleData(
outputVideoTrack, encoderOutputBuffer, videoEncoderOutputBufferInfo
)
}
}
if (videoEncoderOutputBufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM
!= 0
) {
videoEncoderDone = true
}
videoEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false)
break
}
while (!audioEncoderDone
&& (encoderOutputAudioFormat == null || muxing)
) {
val encoderOutputBufferIndex = audioEncoder.dequeueOutputBuffer(
audioEncoderOutputBufferInfo, TIMEOUT_USEC.toLong()
)
if (encoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
break
}
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
audioEncoderOutputBuffers = audioEncoder.outputBuffers
break
}
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
encoderOutputAudioFormat = audioEncoder.outputFormat
break
}
val encoderOutputBuffer: ByteBuffer? =
audioEncoderOutputBuffers!![encoderOutputBufferIndex]
if (audioEncoderOutputBufferInfo.flags and MediaCodec.BUFFER_FLAG_CODEC_CONFIG
!= 0
) {
audioEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false)
break
}
if (audioEncoderOutputBufferInfo.size != 0) encoderOutputBuffer?.let {
muxer.writeSampleData(
outputAudioTrack, it, audioEncoderOutputBufferInfo
)
}
if (audioEncoderOutputBufferInfo.flags and MediaCodec.BUFFER_FLAG_END_OF_STREAM
!= 0
) audioEncoderDone = true
audioEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false)
break
}
if (!muxing && encoderOutputAudioFormat != null
&& encoderOutputVideoFormat != null
) {
outputVideoTrack = muxer.addTrack(encoderOutputVideoFormat)
outputAudioTrack = muxer.addTrack(encoderOutputAudioFormat)
muxer.start()
muxing = true
}
}
}
private fun isVideoFormat(format: MediaFormat): Boolean {
return getMimeTypeFor(format)!!.startsWith("video/")
}
private fun isAudioFormat(format: MediaFormat): Boolean {
return getMimeTypeFor(format)!!.startsWith("audio/")
}
private fun getMimeTypeFor(format: MediaFormat): String? {
return format.getString(MediaFormat.KEY_MIME)
}
private fun selectCodec(mimeType: String): MediaCodecInfo? {
val numCodecs = MediaCodecList.getCodecCount()
for (i in 0 until numCodecs) {
val codecInfo = MediaCodecList.getCodecInfoAt(i)
if (!codecInfo.isEncoder) {
continue
}
val types = codecInfo.supportedTypes
for (j in types.indices) {
if (types[j].equals(mimeType, ignoreCase = true)) {
return codecInfo
}
}
}
return null
}
}
Upvotes: 0
snachmsm
Reputation: 19273
Basing on bigflake.com/mediacodec/ (awesome source of knowledge about Media-classes) I've tried few ways and finally ExtractDecodeEditEncodeMuxTest turned out very helpfull. This test wasn't described in article on bigflake site, but it can be found HERE next to other classes mentioned in text.
So, I've copied most of code from above mentioned ExtractDecodeEditEncodeMuxTest class and there it is: VideoResolutionChanger. It gives me 2Mb HD video from 16 Mb fullHD. Nice! And fast! On my device whole process is a bit longer than input video duration, e.g. 10 secs video input -> 11-12 secs of processing. With ffmpeg-java it would be smth about 40 secs or more (and 9 Mb more for app).
Here we go:
VideoResolutionChanger:
@TargetApi(18)
public class VideoResolutionChanger {
private static final int TIMEOUT_USEC = 10000;
private static final String OUTPUT_VIDEO_MIME_TYPE = "video/avc";
private static final int OUTPUT_VIDEO_BIT_RATE = 2048 * 1024;
private static final int OUTPUT_VIDEO_FRAME_RATE = 30;
private static final int OUTPUT_VIDEO_IFRAME_INTERVAL = 10;
private static final int OUTPUT_VIDEO_COLOR_FORMAT =
MediaCodecInfo.CodecCapabilities.COLOR_FormatSurface;
private static final String OUTPUT_AUDIO_MIME_TYPE = "audio/mp4a-latm";
private static final int OUTPUT_AUDIO_CHANNEL_COUNT = 2;
private static final int OUTPUT_AUDIO_BIT_RATE = 128 * 1024;
private static final int OUTPUT_AUDIO_AAC_PROFILE =
MediaCodecInfo.CodecProfileLevel.AACObjectHE;
private static final int OUTPUT_AUDIO_SAMPLE_RATE_HZ = 44100;
private int mWidth = 1280;
private int mHeight = 720;
private String mOutputFile, mInputFile;
public String changeResolution(File f)
throws Throwable {
mInputFile=f.getAbsolutePath();
String filePath = mInputFile.substring(0, mInputFile.lastIndexOf(File.separator));
String[] splitByDot = mInputFile.split("\\.");
String ext="";
if(splitByDot!=null && splitByDot.length>1)
ext = splitByDot[splitByDot.length-1];
String fileName = mInputFile.substring(mInputFile.lastIndexOf(File.separator)+1,
mInputFile.length());
if(ext.length()>0)
fileName=fileName.replace("."+ext, "_out.mp4");
else
fileName=fileName.concat("_out.mp4");
final File outFile = new File(Environment.getExternalStorageDirectory(), fileName);
if(!outFile.exists())
outFile.createNewFile();
mOutputFile=outFile.getAbsolutePath();
ChangerWrapper.changeResolutionInSeparatedThread(this);
return mOutputFile;
}
private static class ChangerWrapper implements Runnable {
private Throwable mThrowable;
private VideoResolutionChanger mChanger;
private ChangerWrapper(VideoResolutionChanger changer) {
mChanger = changer;
}
@Override
public void run() {
try {
mChanger.prepareAndChangeResolution();
} catch (Throwable th) {
mThrowable = th;
}
}
public static void changeResolutionInSeparatedThread(VideoResolutionChanger changer)
throws Throwable {
ChangerWrapper wrapper = new ChangerWrapper(changer);
Thread th = new Thread(wrapper, ChangerWrapper.class.getSimpleName());
th.start();
th.join();
if (wrapper.mThrowable != null)
throw wrapper.mThrowable;
}
}
private void prepareAndChangeResolution() throws Exception {
Exception exception = null;
MediaCodecInfo videoCodecInfo = selectCodec(OUTPUT_VIDEO_MIME_TYPE);
if (videoCodecInfo == null)
return;
MediaCodecInfo audioCodecInfo = selectCodec(OUTPUT_AUDIO_MIME_TYPE);
if (audioCodecInfo == null)
return;
MediaExtractor videoExtractor = null;
MediaExtractor audioExtractor = null;
OutputSurface outputSurface = null;
MediaCodec videoDecoder = null;
MediaCodec audioDecoder = null;
MediaCodec videoEncoder = null;
MediaCodec audioEncoder = null;
MediaMuxer muxer = null;
InputSurface inputSurface = null;
try {
videoExtractor = createExtractor();
int videoInputTrack = getAndSelectVideoTrackIndex(videoExtractor);
MediaFormat inputFormat = videoExtractor.getTrackFormat(videoInputTrack);
MediaMetadataRetriever m = new MediaMetadataRetriever();
m.setDataSource(mInputFile);
int inputWidth, inputHeight;
try {
inputWidth = Integer.parseInt(m.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_WIDTH));
inputHeight = Integer.parseInt(m.extractMetadata(MediaMetadataRetriever.METADATA_KEY_VIDEO_HEIGHT));
} catch (Exception e) {
Bitmap thumbnail = m.getFrameAtTime();
inputWidth = thumbnail.getWidth();
inputHeight = thumbnail.getHeight();
thumbnail.recycle();
}
if(inputWidth>inputHeight){
if(mWidth<mHeight){
int w = mWidth;
mWidth=mHeight;
mHeight=w;
}
}
else{
if(mWidth>mHeight){
int w = mWidth;
mWidth=mHeight;
mHeight=w;
}
}
MediaFormat outputVideoFormat =
MediaFormat.createVideoFormat(OUTPUT_VIDEO_MIME_TYPE, mWidth, mHeight);
outputVideoFormat.setInteger(
MediaFormat.KEY_COLOR_FORMAT, OUTPUT_VIDEO_COLOR_FORMAT);
outputVideoFormat.setInteger(MediaFormat.KEY_BIT_RATE, OUTPUT_VIDEO_BIT_RATE);
outputVideoFormat.setInteger(MediaFormat.KEY_FRAME_RATE, OUTPUT_VIDEO_FRAME_RATE);
outputVideoFormat.setInteger(
MediaFormat.KEY_I_FRAME_INTERVAL, OUTPUT_VIDEO_IFRAME_INTERVAL);
AtomicReference<Surface> inputSurfaceReference = new AtomicReference<Surface>();
videoEncoder = createVideoEncoder(
videoCodecInfo, outputVideoFormat, inputSurfaceReference);
inputSurface = new InputSurface(inputSurfaceReference.get());
inputSurface.makeCurrent();
outputSurface = new OutputSurface();
videoDecoder = createVideoDecoder(inputFormat, outputSurface.getSurface());
audioExtractor = createExtractor();
int audioInputTrack = getAndSelectAudioTrackIndex(audioExtractor);
MediaFormat inputAudioFormat = audioExtractor.getTrackFormat(audioInputTrack);
MediaFormat outputAudioFormat =
MediaFormat.createAudioFormat(inputAudioFormat.getString(MediaFormat.KEY_MIME),
inputAudioFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE),
inputAudioFormat.getInteger(MediaFormat.KEY_CHANNEL_COUNT));
outputAudioFormat.setInteger(MediaFormat.KEY_BIT_RATE, OUTPUT_AUDIO_BIT_RATE);
outputAudioFormat.setInteger(MediaFormat.KEY_AAC_PROFILE, OUTPUT_AUDIO_AAC_PROFILE);
audioEncoder = createAudioEncoder(audioCodecInfo, outputAudioFormat);
audioDecoder = createAudioDecoder(inputAudioFormat);
muxer = new MediaMuxer(mOutputFile, MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4);
changeResolution(videoExtractor, audioExtractor,
videoDecoder, videoEncoder,
audioDecoder, audioEncoder,
muxer, inputSurface, outputSurface);
} finally {
try {
if (videoExtractor != null)
videoExtractor.release();
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (audioExtractor != null)
audioExtractor.release();
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (videoDecoder != null) {
videoDecoder.stop();
videoDecoder.release();
}
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (outputSurface != null) {
outputSurface.release();
}
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (videoEncoder != null) {
videoEncoder.stop();
videoEncoder.release();
}
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (audioDecoder != null) {
audioDecoder.stop();
audioDecoder.release();
}
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (audioEncoder != null) {
audioEncoder.stop();
audioEncoder.release();
}
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (muxer != null) {
muxer.stop();
muxer.release();
}
} catch(Exception e) {
if (exception == null)
exception = e;
}
try {
if (inputSurface != null)
inputSurface.release();
} catch(Exception e) {
if (exception == null)
exception = e;
}
}
if (exception != null)
throw exception;
}
private MediaExtractor createExtractor() throws IOException {
MediaExtractor extractor;
extractor = new MediaExtractor();
extractor.setDataSource(mInputFile);
return extractor;
}
private MediaCodec createVideoDecoder(MediaFormat inputFormat, Surface surface) throws IOException {
MediaCodec decoder = MediaCodec.createDecoderByType(getMimeTypeFor(inputFormat));
decoder.configure(inputFormat, surface, null, 0);
decoder.start();
return decoder;
}
private MediaCodec createVideoEncoder(MediaCodecInfo codecInfo, MediaFormat format,
AtomicReference<Surface> surfaceReference) throws IOException {
MediaCodec encoder = MediaCodec.createByCodecName(codecInfo.getName());
encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
surfaceReference.set(encoder.createInputSurface());
encoder.start();
return encoder;
}
private MediaCodec createAudioDecoder(MediaFormat inputFormat) throws IOException {
MediaCodec decoder = MediaCodec.createDecoderByType(getMimeTypeFor(inputFormat));
decoder.configure(inputFormat, null, null, 0);
decoder.start();
return decoder;
}
private MediaCodec createAudioEncoder(MediaCodecInfo codecInfo, MediaFormat format) throws IOException {
MediaCodec encoder = MediaCodec.createByCodecName(codecInfo.getName());
encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
encoder.start();
return encoder;
}
private int getAndSelectVideoTrackIndex(MediaExtractor extractor) {
for (int index = 0; index < extractor.getTrackCount(); ++index) {
if (isVideoFormat(extractor.getTrackFormat(index))) {
extractor.selectTrack(index);
return index;
}
}
return -1;
}
private int getAndSelectAudioTrackIndex(MediaExtractor extractor) {
for (int index = 0; index < extractor.getTrackCount(); ++index) {
if (isAudioFormat(extractor.getTrackFormat(index))) {
extractor.selectTrack(index);
return index;
}
}
return -1;
}
private void changeResolution(MediaExtractor videoExtractor, MediaExtractor audioExtractor,
MediaCodec videoDecoder, MediaCodec videoEncoder,
MediaCodec audioDecoder, MediaCodec audioEncoder,
MediaMuxer muxer,
InputSurface inputSurface, OutputSurface outputSurface) {
ByteBuffer[] videoDecoderInputBuffers = null;
ByteBuffer[] videoDecoderOutputBuffers = null;
ByteBuffer[] videoEncoderOutputBuffers = null;
MediaCodec.BufferInfo videoDecoderOutputBufferInfo = null;
MediaCodec.BufferInfo videoEncoderOutputBufferInfo = null;
videoDecoderInputBuffers = videoDecoder.getInputBuffers();
videoDecoderOutputBuffers = videoDecoder.getOutputBuffers();
videoEncoderOutputBuffers = videoEncoder.getOutputBuffers();
videoDecoderOutputBufferInfo = new MediaCodec.BufferInfo();
videoEncoderOutputBufferInfo = new MediaCodec.BufferInfo();
ByteBuffer[] audioDecoderInputBuffers = null;
ByteBuffer[] audioDecoderOutputBuffers = null;
ByteBuffer[] audioEncoderInputBuffers = null;
ByteBuffer[] audioEncoderOutputBuffers = null;
MediaCodec.BufferInfo audioDecoderOutputBufferInfo = null;
MediaCodec.BufferInfo audioEncoderOutputBufferInfo = null;
audioDecoderInputBuffers = audioDecoder.getInputBuffers();
audioDecoderOutputBuffers = audioDecoder.getOutputBuffers();
audioEncoderInputBuffers = audioEncoder.getInputBuffers();
audioEncoderOutputBuffers = audioEncoder.getOutputBuffers();
audioDecoderOutputBufferInfo = new MediaCodec.BufferInfo();
audioEncoderOutputBufferInfo = new MediaCodec.BufferInfo();
MediaFormat decoderOutputVideoFormat = null;
MediaFormat decoderOutputAudioFormat = null;
MediaFormat encoderOutputVideoFormat = null;
MediaFormat encoderOutputAudioFormat = null;
int outputVideoTrack = -1;
int outputAudioTrack = -1;
boolean videoExtractorDone = false;
boolean videoDecoderDone = false;
boolean videoEncoderDone = false;
boolean audioExtractorDone = false;
boolean audioDecoderDone = false;
boolean audioEncoderDone = false;
int pendingAudioDecoderOutputBufferIndex = -1;
boolean muxing = false;
while ((!videoEncoderDone) || (!audioEncoderDone)) {
while (!videoExtractorDone
&& (encoderOutputVideoFormat == null || muxing)) {
int decoderInputBufferIndex = videoDecoder.dequeueInputBuffer(TIMEOUT_USEC);
if (decoderInputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER)
break;
ByteBuffer decoderInputBuffer = videoDecoderInputBuffers[decoderInputBufferIndex];
int size = videoExtractor.readSampleData(decoderInputBuffer, 0);
long presentationTime = videoExtractor.getSampleTime();
if (size >= 0) {
videoDecoder.queueInputBuffer(
decoderInputBufferIndex,
0,
size,
presentationTime,
videoExtractor.getSampleFlags());
}
videoExtractorDone = !videoExtractor.advance();
if (videoExtractorDone)
videoDecoder.queueInputBuffer(decoderInputBufferIndex,
0, 0, 0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
break;
}
while (!audioExtractorDone
&& (encoderOutputAudioFormat == null || muxing)) {
int decoderInputBufferIndex = audioDecoder.dequeueInputBuffer(TIMEOUT_USEC);
if (decoderInputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER)
break;
ByteBuffer decoderInputBuffer = audioDecoderInputBuffers[decoderInputBufferIndex];
int size = audioExtractor.readSampleData(decoderInputBuffer, 0);
long presentationTime = audioExtractor.getSampleTime();
if (size >= 0)
audioDecoder.queueInputBuffer(decoderInputBufferIndex, 0, size,
presentationTime, audioExtractor.getSampleFlags());
audioExtractorDone = !audioExtractor.advance();
if (audioExtractorDone)
audioDecoder.queueInputBuffer(decoderInputBufferIndex, 0, 0,
0, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
break;
}
while (!videoDecoderDone
&& (encoderOutputVideoFormat == null || muxing)) {
int decoderOutputBufferIndex =
videoDecoder.dequeueOutputBuffer(
videoDecoderOutputBufferInfo, TIMEOUT_USEC);
if (decoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER)
break;
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
videoDecoderOutputBuffers = videoDecoder.getOutputBuffers();
break;
}
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
decoderOutputVideoFormat = videoDecoder.getOutputFormat();
break;
}
ByteBuffer decoderOutputBuffer =
videoDecoderOutputBuffers[decoderOutputBufferIndex];
if ((videoDecoderOutputBufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG)
!= 0) {
videoDecoder.releaseOutputBuffer(decoderOutputBufferIndex, false);
break;
}
boolean render = videoDecoderOutputBufferInfo.size != 0;
videoDecoder.releaseOutputBuffer(decoderOutputBufferIndex, render);
if (render) {
outputSurface.awaitNewImage();
outputSurface.drawImage();
inputSurface.setPresentationTime(
videoDecoderOutputBufferInfo.presentationTimeUs * 1000);
inputSurface.swapBuffers();
}
if ((videoDecoderOutputBufferInfo.flags
& MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0) {
videoDecoderDone = true;
videoEncoder.signalEndOfInputStream();
}
break;
}
while (!audioDecoderDone && pendingAudioDecoderOutputBufferIndex == -1
&& (encoderOutputAudioFormat == null || muxing)) {
int decoderOutputBufferIndex =
audioDecoder.dequeueOutputBuffer(
audioDecoderOutputBufferInfo, TIMEOUT_USEC);
if (decoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER)
break;
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
audioDecoderOutputBuffers = audioDecoder.getOutputBuffers();
break;
}
if (decoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
decoderOutputAudioFormat = audioDecoder.getOutputFormat();
break;
}
ByteBuffer decoderOutputBuffer =
audioDecoderOutputBuffers[decoderOutputBufferIndex];
if ((audioDecoderOutputBufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG)
!= 0) {
audioDecoder.releaseOutputBuffer(decoderOutputBufferIndex, false);
break;
}
pendingAudioDecoderOutputBufferIndex = decoderOutputBufferIndex;
break;
}
while (pendingAudioDecoderOutputBufferIndex != -1) {
int encoderInputBufferIndex = audioEncoder.dequeueInputBuffer(TIMEOUT_USEC);
ByteBuffer encoderInputBuffer = audioEncoderInputBuffers[encoderInputBufferIndex];
int size = audioDecoderOutputBufferInfo.size;
long presentationTime = audioDecoderOutputBufferInfo.presentationTimeUs;
if (size >= 0) {
ByteBuffer decoderOutputBuffer =
audioDecoderOutputBuffers[pendingAudioDecoderOutputBufferIndex]
.duplicate();
decoderOutputBuffer.position(audioDecoderOutputBufferInfo.offset);
decoderOutputBuffer.limit(audioDecoderOutputBufferInfo.offset + size);
encoderInputBuffer.position(0);
encoderInputBuffer.put(decoderOutputBuffer);
audioEncoder.queueInputBuffer(
encoderInputBufferIndex,
0,
size,
presentationTime,
audioDecoderOutputBufferInfo.flags);
}
audioDecoder.releaseOutputBuffer(pendingAudioDecoderOutputBufferIndex, false);
pendingAudioDecoderOutputBufferIndex = -1;
if ((audioDecoderOutputBufferInfo.flags
& MediaCodec.BUFFER_FLAG_END_OF_STREAM) != 0)
audioDecoderDone = true;
break;
}
while (!videoEncoderDone
&& (encoderOutputVideoFormat == null || muxing)) {
int encoderOutputBufferIndex = videoEncoder.dequeueOutputBuffer(
videoEncoderOutputBufferInfo, TIMEOUT_USEC);
if (encoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER)
break;
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
videoEncoderOutputBuffers = videoEncoder.getOutputBuffers();
break;
}
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
encoderOutputVideoFormat = videoEncoder.getOutputFormat();
break;
}
ByteBuffer encoderOutputBuffer =
videoEncoderOutputBuffers[encoderOutputBufferIndex];
if ((videoEncoderOutputBufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG)
!= 0) {
videoEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false);
break;
}
if (videoEncoderOutputBufferInfo.size != 0) {
muxer.writeSampleData(
outputVideoTrack, encoderOutputBuffer, videoEncoderOutputBufferInfo);
}
if ((videoEncoderOutputBufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM)
!= 0) {
videoEncoderDone = true;
}
videoEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false);
break;
}
while (!audioEncoderDone
&& (encoderOutputAudioFormat == null || muxing)) {
int encoderOutputBufferIndex = audioEncoder.dequeueOutputBuffer(
audioEncoderOutputBufferInfo, TIMEOUT_USEC);
if (encoderOutputBufferIndex == MediaCodec.INFO_TRY_AGAIN_LATER) {
break;
}
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
audioEncoderOutputBuffers = audioEncoder.getOutputBuffers();
break;
}
if (encoderOutputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
encoderOutputAudioFormat = audioEncoder.getOutputFormat();
break;
}
ByteBuffer encoderOutputBuffer =
audioEncoderOutputBuffers[encoderOutputBufferIndex];
if ((audioEncoderOutputBufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG)
!= 0) {
audioEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false);
break;
}
if (audioEncoderOutputBufferInfo.size != 0)
muxer.writeSampleData(
outputAudioTrack, encoderOutputBuffer, audioEncoderOutputBufferInfo);
if ((audioEncoderOutputBufferInfo.flags & MediaCodec.BUFFER_FLAG_END_OF_STREAM)
!= 0)
audioEncoderDone = true;
audioEncoder.releaseOutputBuffer(encoderOutputBufferIndex, false);
break;
}
if (!muxing && (encoderOutputAudioFormat != null)
&& (encoderOutputVideoFormat != null)) {
outputVideoTrack = muxer.addTrack(encoderOutputVideoFormat);
outputAudioTrack = muxer.addTrack(encoderOutputAudioFormat);
muxer.start();
muxing = true;
}
}
}
private static boolean isVideoFormat(MediaFormat format) {
return getMimeTypeFor(format).startsWith("video/");
}
private static boolean isAudioFormat(MediaFormat format) {
return getMimeTypeFor(format).startsWith("audio/");
}
private static String getMimeTypeFor(MediaFormat format) {
return format.getString(MediaFormat.KEY_MIME);
}
private static MediaCodecInfo selectCodec(String mimeType) {
int numCodecs = MediaCodecList.getCodecCount();
for (int i = 0; i < numCodecs; i++) {
MediaCodecInfo codecInfo = MediaCodecList.getCodecInfoAt(i);
if (!codecInfo.isEncoder()) {
continue;
}
String[] types = codecInfo.getSupportedTypes();
for (int j = 0; j < types.length; j++) {
if (types[j].equalsIgnoreCase(mimeType)) {
return codecInfo;
}
}
}
return null;
}
}
it needs also InputSurface, OutputSurface and TextureRender, which are placed next to ExtractDecodeEditEncodeMuxTest (above HERE link). Put these three in same package with VideoResolutionChanger and use it like this:
try{
String pathToReEncodedFile =
new VideoResolutionChanger().changeResolution(videoFilePath);
}catch(Throwable t){/* smth wrong :( */}
Where videoFilePath might be obtained from File using file.getAbsolutePath().
I know it's not the cleanest and probably not most-effective/efficient way, but I've been looking for similar code for last two days and found lot of topics, which most redirected me to INDE, ffmpeg or jcodec, other were left without proper answer. So I'm leaving it here, use this wisely!
LIMITATIONS:
- above use-it-like-this snippet can not be started in main looper thread (ui), e.g. straight inside
Activity. best way is to createIntentServiceand pass input file pathStringinIntents extraBundle. Then you can runchangeResolutionstright insideonHandleIntent; - API18 and above (
MediaMuxerintroduced); - API18 needs of course
WRITE_EXTERNAL_STORAGE, API19 and above has this "built-in";
@fadden THANK YOU for your work and support! :)
Upvotes: 22
Sepehr GH
Reputation: 1397
I'm not going to mind the implementation and coding problems of the question. But we have gone through the same disaster as ffmpeg increased our application size for 19MB at least, and I was using this stackoverflow question to come up with a library that does the same without ffmpeg. Apparently guys at linkedin have done it before. Check this article.
The project is called LiTr and is available on github. It uses android MediaCodec and MediaMuxer so you can refer to the codes to get help with your own projects if you need to. This question was asked 4 years ago but I hope this helps someone now.
Upvotes: 1
fadden
Reputation: 52353
The MediaMuxer is not involved in the compression or scaling of video. All it does is take the H.264 output from MediaCodec and wrap it in a .mp4 file wrapper.
Looking at your code, you're extracting NAL units with MediaExtractor and immediately re-wrapping them with MediaMuxer. This should be extremely fast and have no impact on the video itself, as you're just re-wrapping the H.264.
To scale the video you need to decode the video with a MediaCodec decoder, feeding the NAL units from MediaExtractor into it, and re-encode it with a MediaCodec encoder, passing the frames to a MediaMuxer.
You've found bigflake.com; see also Grafika. Neither of these has exactly what you're looking for, but the various pieces are there.
It's best to decode to a Surface, not a ByteBuffer. This requires API 18, but for sanity it's best to forget that MediaCodec existed before then. And you'll need API 18 for MediaMuxer anyway.
Upvotes: 5
Marlon
Reputation: 1473
You can try Intel INDE Media for Mobile, tutorials on https://software.intel.com/en-us/articles/intel-inde-media-pack-for-android-tutorials. It has a sample that shows how to use it to transcode=recompress video files.
You can set smaller resolution and\or bitrate to output to get smaller file https://github.com/INDExOS/media-for-mobile/blob/master/Android/samples/apps/src/com/intel/inde/mp/samples/ComposerTranscodeCoreActivity.java
Upvotes: 0
Related Questions
- Android Change Resolution of Video File
- How to compress mp4 video using MediaCodec Android?
- Android make mp4 video file smaller
- Compressing video file based with Multiple Resolution format options in Android
- Transcode video to lower bitrate and stream
- Android MediaCodec: Reduce mp4 video size
- Using MediaCodec to compress video
- Android video and image resizing
- Compress Videos using android MediaCodec api
- Reduce video size/quality in android