Why am I getting "Failed to set remote video description send parameters for m-section with mid='0'" error in Android when it works in Chrome?

Question

Background Information

Hi there, I am attempting to build a WebRTC client in Android that subscribes to a video feed that is being broadcast using NodeJS and JavaScript.

The broadcaster code can be viewed in its entirety in this lovely article by Gabriel Tanner.

It works beautifully when running it in localhost under the http://localhost:4000/broadcaster.html in Chrome and then visiting my IP Address from another device on the network. I can see the video and it is near real time.

I have tried this using two different webcam devices, both a built-in and a USB webcam but the Android client does not work even though the JavaScript Broadcaster and Client works fine.

The task at hand

After following the tutorial and getting the example to work I decided to try and implement my own Android application for which the entire source code can be viewed right here on my GitHub.

I have followed various tutorials around the place and the issue always stems from attempting to set the remote description which is done with the following bit of code:

    private void setRemoteDescription(Object[] arguments) {
        JSONObject message = (JSONObject) arguments[1];
        try {
            String sdp = message.getString("sdp");
            SessionDescription sessionDescription = new SessionDescription(OFFER, sdp);
            peerConnection.setRemoteDescription(new SimpleSdpObserver(), sessionDescription);
        } catch (JSONException e) {
            Log.e(TAG, "setRemoteDescription: failed to parse JSON", e);
        }
    }

Like I said the full code can be viewed in the GitHub and I don't want to clutter this post too much but essentially as per Gabriel's tutorial when the socket emits "watcher" upon connection (which the Android App does correctly) it will then send back an "offer" which is then listened for on the client side like this:

    private void bindSocketEvents() {
        socket.on(EVENT_CONNECT, args -> {
            socket.emit("watcher");
        }).on("broadcaster", args -> {
            socket.emit("watcher");
        }).on("offer", args -> {
            setRemoteDescription(args);
            performAnswer();
        }).on("candidate", this::addIceCandidate);
    }

Where we then see that upon "offer" we call the code with the args from JavaScript and attempts to set the Remote Description but it fails to do so.

This is an example SDP that I get from triggering this:

v=0
o=- 7040957491050894781 2 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS l91G3Ekmme9tlvEso1ApTqS6djaxtsamgfLu
m=video 9 UDP/TLS/RTP/SAVPF 96 97 98 99 100 101 102 121 127 120 125 107 108 109 35 36 124 119 123
c=IN IP4 0.0.0.0
a=rtcp:9 IN IP4 0.0.0.0
a=ice-ufrag:XEnc
a=ice-pwd:2PyQmLOm9YPPf1Pozvonticd
a=ice-options:trickle
a=fingerprint:sha-256 64:22:D7:93:FD:6C:A9:94:E3:65:76:B0:DB:4E:E9:8E:91:46:56:87:B1:E3:E9:B3:24:D0:CF:A5:3F:91:0A:FD
a=setup:actpass
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendrecv
a=msid:l91G3Ekmme9tlvEso1ApTqS6djaxtsamgfLu 7eb4296c-f3e4-4e95-9f6f-05aa386a3176
a=rtcp-mux
a=rtcp-rsize
a=rtpmap:96 VP8/90000
a=rtcp-fb:96 goog-remb
a=rtcp-fb:96 transport-cc
a=rtcp-fb:96 ccm fir
a=rtcp-fb:96 nack
a=rtcp-fb:96 nack pli
a=rtpmap:97 rtx/90000
a=fmtp:97 apt=96
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=rtpmap:99 rtx/90000
a=fmtp:99 apt=98
a=rtpmap:100 VP9/90000
a=rtcp-fb:100 goog-remb
a=rtcp-fb:100 transport-cc
a=rtcp-fb:100 ccm fir
a=rtcp-fb:100 nack
a=rtcp-fb:100 nack pli
a=fmtp:100 profile-id=2
a=rtpmap:101 rtx/90000
a=fmtp:101 apt=100
a=rtpmap:102 H264/90000
a=rtcp-fb:102 goog-remb
a=rtcp-fb:102 transport-cc
a=rtcp-fb:102 ccm fir
a=rtcp-fb:102 nack
a=rtcp-fb:102 nack pli
a=fmtp:102 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42001f
a=rtpmap:121 rtx/90000
a=fmtp:121 apt=102
a=rtpmap:127 H264/90000
a=rtcp-fb:127 goog-remb
a=rtcp-fb:127 transport-cc
a=rtcp-fb:127 ccm fir
a=rtcp-fb:127 nack
a=rtcp-fb:127 nack pli
a=fmtp:127 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42001f
a=rtpmap:120 rtx/90000
a=fmtp:120 apt=127
a=rtpmap:125 H264/90000
a=rtcp-fb:125 goog-remb
a=rtcp-fb:125 transport-cc
a=rtcp-fb:125 ccm fir
a=rtcp-fb:125 nack
a=rtcp-fb:125 nack pli
a=fmtp:125 level-asymmetry-allowed=1;packetization-mode=1;profile-level-id=42e01f
a=rtpmap:107 rtx/90000
a=fmtp:107 apt=125
a=rtpmap:108 H264/90000
a=rtcp-fb:108 goog-remb
a=rtcp-fb:108 transport-cc
a=rtcp-fb:108 ccm fir
a=rtcp-fb:108 nack
a=rtcp-fb:108 nack pli
a=fmtp:108 level-asymmetry-allowed=1;packetization-mode=0;profile-level-id=42e01f
a=rtpmap:109 rtx/90000
a=fmtp:109 apt=108
a=rtpmap:35 AV1X/90000
a=rtcp-fb:35 goog-remb
a=rtcp-fb:35 transport-cc
a=rtcp-fb:35 ccm fir
a=rtcp-fb:35 nack
a=rtcp-fb:35 nack pli
a=rtpmap:36 rtx/90000
a=fmtp:36 apt=35
a=rtpmap:124 red/90000
a=rtpmap:119 rtx/90000
a=fmtp:119 apt=124
a=rtpmap:123 ulpfec/90000
a=ssrc-group:FID 4173041010 2470630943
a=ssrc:4173041010 cname:VrEB0NYLt23UUzRD
a=ssrc:4173041010 msid:l91G3Ekmme9tlvEso1ApTqS6djaxtsamgfLu 7eb4296c-f3e4-4e95-9f6f-05aa386a3176
a=ssrc:4173041010 mslabel:l91G3Ekmme9tlvEso1ApTqS6djaxtsamgfLu
a=ssrc:4173041010 label:7eb4296c-f3e4-4e95-9f6f-05aa386a3176
a=ssrc:2470630943 cname:VrEB0NYLt23UUzRD
a=ssrc:2470630943 msid:l91G3Ekmme9tlvEso1ApTqS6djaxtsamgfLu 7eb4296c-f3e4-4e95-9f6f-05aa386a3176
a=ssrc:2470630943 mslabel:l91G3Ekmme9tlvEso1ApTqS6djaxtsamgfLu
a=ssrc:2470630943 label:7eb4296c-f3e4-4e95-9f6f-05aa386a3176

The error

Now the thing is that when I try and set this remote description these errors get thrown and we never get to attempt to peerConnection.createAnswer:

2021-11-12 13:45:52.819 26795-27780/au.com.australiandroid.androidclient D/CustomPeerConnectionObs: onSignalingChange: new signaling state is HAVE_REMOTE_OFFER
2021-11-12 13:45:52.820 26795-27780/au.com.australiandroid.androidclient E/SimpleSdpObserver: onSetFailure: failed to set the remote description: Failed to set remote offer sdp: Failed to set remote video description send parameters for m-section with mid='0'.
2021-11-12 13:45:52.820 26795-27780/au.com.australiandroid.androidclient E/MainActivity: onCreateFailure: failed to create answer: Session error code: ERROR_CONTENT. Session error description: Failed to set remote video description send parameters for m-section with mid='0'..

If you have any information whatsoever as to why it fails to setRemoteDescription or even if you have any advice regarding how to implement an Android WebRTC client then anything would be helpful at this point. I have struggled for weeks and I don't know where to go from here because Googling this error isn't helping. Either because I simply do not understand the answer or because the answer isn't relevant. I don't know but please don't just paste a link to the top hit on Google because I have read so many of them.

Answer 1

The error message was triggered due to the offer containing H264 codecs whilst the Android Client was not anticipating H264 and was not setup to encode and/or decode this particular hardware encoded stream.

The fix was to ensure that the connection factory was setup as such:

    private void createPeerConnectionFactory() {
        PeerConnectionFactory.InitializationOptions initializationOptions =
                PeerConnectionFactory.InitializationOptions.builder(this)
                        .createInitializationOptions();

        PeerConnectionFactory.initialize(initializationOptions);
        PeerConnectionFactory.Options options = new PeerConnectionFactory.Options();

        VideoEncoderFactory encoderFactory = new DefaultVideoEncoderFactory(rootEglBase.getEglBaseContext(), true, true);
        VideoDecoderFactory decoderFactory = new DefaultVideoDecoderFactory(rootEglBase.getEglBaseContext());

        peerConnectionFactory = PeerConnectionFactory.builder()
                .setOptions(options)
                .setVideoDecoderFactory(decoderFactory)
                .setVideoEncoderFactory(encoderFactory)
                .createPeerConnectionFactory();

        List<PeerConnection.IceServer> iceServers = new ArrayList<>();
        PeerConnection.RTCConfiguration rtcConfig = new PeerConnection.RTCConfiguration(iceServers);
        createPeerConnection(rtcConfig);
    }

Where the rootEglBase was setup earlier by defining a variable in the activity like private EglBase rootEglBase; and following that up with creating a new instance of it with EglBase.create() in the Activities onCreate method.

The actual winning lines in the peerConnectionFactory was defining the encoderFactory and the decoderFactory and specifying the correct flags to tell WebRTC that we want to use h264HighProfile.

This is seen in this line: VideoEncoderFactory encoderFactory = new DefaultVideoEncoderFactory(rootEglBase.getEglBaseContext(), true, true);

Where the first true boolean is to enable VP8 encoding and the second is for H264.

Once this was set correctly it would accept the offer when attempting to setRemoteDescription and it lead me down the right track in order to solve many of the other problems that were still in my code.

So if you are getting that same error message chances are it's because of some form of hardware encoding possibly to do with H264.

If you find this question and my answer and it not quite making sense then just shoot me a PM or comment on this answer and I'll try my best to help you get it working as well.