CODEWITHSUNDEEP
androidvideo-streamingh.264rtmp
user3849009
user3849009

Reputation: 11

Streaming video of android surface

I have a problem with rtmp streaming of android surface to a client application. My solution has a very big latency, because my surface is not producing frames 60 times a second, it can produce it in any time (once in 30 seconds for example). So I want to show each new produced frame to the client immediately.

Android is pushing every frame, it looks fine. Client app (jwplayer or vlc) receives it, but it waiting for something. It becomes showing video only after receiving a number of frames. But I need to see every incoming frame on the client side when it just have been received.

How it is working now: I have a Surface object, obtained from MediaCodec class. MediaCodec is set for h264 video encoding.

MediaCodec mEncoder;

.....

MediaFormat format = MediaFormat.createVideoFormat("video/avc", width, height);
format.setInteger(MediaFormat.KEY_COLOR_FORMAT, colorFormat);
format.setInteger(MediaFormat.KEY_BIT_RATE, videoBitrate);
format.setInteger(MediaFormat.KEY_FRAME_RATE, videoFramePerSecond);
format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, iframeInterval);

try {
    mEncoder = MediaCodec.createEncoderByType("video/avc");
} catch (IOException e) {
    e.printStackTrace();
}
mEncoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
mSurface = mEncoder.createInputSurface();
if (mSurfaceCallback!=null)
    mSurfaceCallback.onSurfaceCreated(mSurface);
mEncoder.start();

Sometimes android is drawing to the surface. I can't control the rate of this drawings. Also I can't draw anything to that surface. When something is changed on the surface, MediaCodec is producing new byteBuffer with h264 frame. I send this frame by rtmp.

On a client side I have html page with jwplayer 

<pre id="myElement"></pre>
<script>
var playerInstance = jwplayer("myElement");
playerInstance.setup({
file:"rtmp://127.0.0.1:1935/live/stream",
height: 800,
width: 480,
autostart: true,
controls: false,
    rtmp: {
        bufferlength: 0.1
    }
});
</script>

I've tried to change iframeInterval, fps of encoding, bufferlength.. Nothing is really helpful.

Is there is any possibility to show incomming frames immeditely?

Upvotes: 1

Views: 1457

Answers (2)

Alex Cohn
Alex Cohn

Reputation: 57203

You can hopefully generate video frames at constant rate, even more than 20 fps to produce smooth video with acceptable latency. h264 encoder will handle a stable picture (one changing once in ~30 sec) gracefully, and when there is no change, frame size will be minimal.

Upvotes: 0

Alexander Pronin
Alexander Pronin

Reputation: 43

What do you mean? If I understood right - you have: vlc(client) ---- rtmp protocol ---- android (producer) You encode video from something (may be camera) using MediaCodec and in vlc there is time latency? right?

At first - what are you using - direct input buffer or MediaCodec.Callback() ? In callback - you can check every frame in onOutputBufferAvailable and calculate time from one frame to another - this will show you - is this problem on android side.

Then you can try to resolve frame transef problem

  1. You can use WireShark to determine frame sending timing and chek - may be this is network problem

  2. Than - vlc and other players try to fill some internal buffer and only after this starting to show video. Try to turn of vlc buffer (https://forum.videolan.org/viewtopic.php?t=40408). Then - there is common that vlc waiting for IDR frame. You can set interval for sending IDR frames in code

    format.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, iframeInterval); iframeInterval in seconds (try to set 1 second) (this will increase streaming size)

Sorry for my bad english

Upvotes: 1

Related Questions