CODEWITHSUNDEEP
androidreact-nativevideoffmpegvideo-streaming
lblenner
lblenner

Reputation: 452

Low latency video player on android

I'd like to be able to stream the video from my webcam to an Android app with a latency below 500ms, on my local network.

To capture and send the video over the network, I use ffmpeg.

ffmpeg -f v4l2 -i /dev/video0 -preset ultrafast -tune zerolatency -vcodec libx264 -an -vf format=yuv420p -f mpegts  udp://192.168.1.155:5000

This command takes the webcam as an input, convert it and send it to a device using the mpegts protocol.
This is not a requirement, if another technique could work, I could change the way I send the video.

I am able to read the video on another PC from the local network with a latency below 500 ms, using commands like

gst-launch-1.0 -v udpsrc port=5000 ! video/mpegts ! tsdemux ! h264parse ! avdec_h264 ! fpsdisplaysink sync=false

or

mpv udp://0.0.0.0:5000 --no-cache --untimed --no-demuxer-thread --video-sync=audio --vd-lavc-threads=1

So it is possible to have this range of latency.
I'd like to have the same thing on Android.

Here are my tries to do that.

Exoplayer

After looking at the different players available on Android studio, it seems like Exoplayer is the go-to choice.
I tried different options indicated in the live-streaming documentation, but I always end up with a stream taking seconds to start and with a latency of seconds.
I tried to add a Button to seek to the default position of the windows, but it results in a loading of several seconds.

DefaultExtractorsFactory extractorsFactory =
                new DefaultExtractorsFactory()
                        .setTsExtractorFlags(DefaultTsPayloadReaderFactory.FLAG_IGNORE_AAC_STREAM);

        player = new SimpleExoPlayer.Builder(this)
                .setMediaSourceFactory(
                        new DefaultMediaSourceFactory(this, extractorsFactory))
                .setLoadControl(new DefaultLoadControl.Builder()
                        .setBufferDurationsMs(DefaultLoadControl.DEFAULT_MIN_BUFFER_MS, DefaultLoadControl.DEFAULT_MAX_BUFFER_MS, 200, 200)
                        .build())
                .build();
        MyPlayerView playerView = findViewById(R.id.player_view);
        // Bind the player to the view.
        playerView.setPlayer(player);
        // Build the media item.
        MediaItem mediaItem = new MediaItem.Builder()
                .setUri(Uri.parse("udp://0.0.0.0:5000"))
                .setLiveMaxOffsetMs(500)
                .setLiveTargetOffsetMs(0)
                .setLiveMinOffsetMs(0)
                .build();
        // Set the media item to be played.
        player.setMediaItem(mediaItem);
        // Prepare the player.
        player.setPlayWhenReady(true);
        player.prepare();
        //player.seekToDefaultPosition();

This issue is about the same issue and the conclusion was that Exoplayer was not fit for this use case.

I'll be honest, ultra low-latency like this isn't ExoPlayer's main use-case

Vlc

Another try was to use the Vlc library.
But I was unable to have the same low latency stream as with the two previous players with Vlc.
I tried changing the preferences of Vlc to stream as fast as possible as described here

Input/Codecs -> x264 preset: ultrafast - zerolatency
Input/Codecs -> Access Module: UDP input
Input/Codecs -> Clock Jitter: 500
Audio: disable audio

I also tried reducing the different buffers.
However, I still have a latency of more than 1 seconds with that.

Gstreamer

Another try was to create a react-native project to use the different players available here.
One player that seemed promising was react-native-gstreamer because it uses gstreamer which is able to stream with low latency (gst-launch command).
But the library is now outdated.

Question

There were other tries, but none were successful.
Is there a problem with one of my approaches ?
And if not, Is there a player on Android (that I missed) which is able to achieve low latency stream like gstream or mpv on linux ?

Upvotes: 2

Views: 2324

Answers (1)

MaloKerrand
MaloKerrand

Reputation: 36

I do not know a native low latency player in Android.
However you can use a WebView in Android Studio and use a player in the web.
With this solution I streamed the webcam of my pc to my phone (in the local network) with livecam.
They use websockets to transmit the video frame by frame, which is not ideal. With this method I had 370 ms of latency.

Upvotes: 2

Related Questions