CODEWITHSUNDEEP
javascriptc++google-chromewebrtclibdatachannel
oarfish
oarfish

Reputation: 4631

WebRTC: Packets get received, but video is not showing

I have a "mediaserver" built with C++ and libdatachannel to receive video streams via WebRTC. It serves a simple signalling protocol, and the client has to send an offer to receive video, and will then get VP9 packets sent over the peer connection. Receiving the stream works fine in my C++ client, but I cannot get it to work in the browser, neither Chrome nor Firefox. The below info is all based on Chrome.

When the server is running locally, the browser client seems to connect, but according to chrome://webrtc-internals/ it seems that no packets get actually received, at least there's nothing in the Stats tables section for the succeeded candidate pair.

enter image description here enter image description here

When the server is running remotely, the browser client seems to connect, and I see that bytes are received:

enter image description here enter image description here

However, in both cases, no video is showing (the promise returned by play() never resolves) and I can't figure out where exactly the problem is. The JS code attempting the connection is this:

<!doctype html>
<title>test js browser client</title>
<section class="content">

    <title>WebRTC test with WebSocket signalling</title>
    <video id="video-element" muted height="200" width="300"></video>

    <p>Mediaserver IP</p>
    <input type="text" id="ip_input" name="input" required minlength="7" size="10" value="46.4.38.241"/>

    <p>Topic</p>
    <input type="text" id="topic_input" name="input" required minlength="1" size="10" value="foobar"/>

    <input type="button" id="button" value="Start stream" />
    <p>Status: <div id="errmsg"></div></p>

    <script>
        document.getElementById("button").onclick=function() {
            const topic = document.getElementById('topic_input').value;
            const mediaserver_ip = document.getElementById('ip_input').value;
            if (topic == "") {
                return
            }

            const videoElement = document.getElementById('video-element');

            // make a new peer connection
            const pc = new RTCPeerConnection({
                // Recommended for libdatachannel
                bundlePolicy: 'max-bundle',
            });

            pc.addTransceiver('video', { direction: 'recvonly' });
            if (!RTCRtpSender.getCapabilities) {
                console.log("RTCRtpSender has no capabilities")
                return
            };

            const { codecs } = RTCRtpSender.getCapabilities('video');
            const [ transceiver ] = pc.getTransceivers();

            const vp9_profile_2 = codecs.find(c => c.mimeType === 'video/VP9' );
            if (vp9_profile_2 && transceiver.setCodecPreferences) {
                transceiver.setCodecPreferences([ vp9_profile_2 ]);
            } else {
                console.log("No vp9 profile or cannot set preferences")
            }
            pc.createOffer({offerToReceiveVideo:1}).then((offer) => {
                console.log("Local offer: ", offer)
                pc.setLocalDescription(offer);

                var websock = new WebSocket("ws://" + mediaserver_ip + ":8080");

                window.onbeforeunload = function() {
                    pc.close();
                    websock.close();
                    return null;
                }

                websock.onopen = function(evt) {
                    websock.send(JSON.stringify({"sdp": pc.localDescription["sdp"], "topic": topic,
                        "direction": "receiver"}));
                }

                websock.onmessage = function(evt) {
                    reply = JSON.parse(evt.data)
                    console.log(`Server reply: ${evt.data}`)
                    const status = reply["ok"]
                    if (status != true) {
                        document.getElementById("topic_input").style.border = "1px solid #FF0000"
                        document.getElementById("errmsg").value = reply["error"]
                        return
                    } else
                    {
                        document.getElementById("topic_input").style.border = "1px solid #00FF00"
                    }
                    console.log("Setting remote response: ", reply["sdp"]);
                    reply.type = "answer"
                    pc.setRemoteDescription(new RTCSessionDescription(reply));
                };


            });

            pc.ontrack = (evt) => {
                const videoElement = document.getElementById('video-element');
                videoElement.srcObject = evt.streams[0];
                videoElement.play();
            };

        };
    </script>
</section>

As an example, the browser's SDP received by the server is

v=0
o=- 8335876533502472473 0 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=group:LS 0
a=msid-semantic:WMS *
a=ice-options:trickle
a=fingerprint:sha-256 7A:E0:F7:6B:15:A7:DE:EA:B6:35:3F:44:DA:F1:F4:7B:CD:D4:24:61:FB:81:4D:D4:4D:5A:CE:E2:AA:3B:C8:28
a=group:BUNDLE 0
a=group:LS 0
a=msid-semantic:WMS *
a=group:BUNDLE 0
a=extmap-allow-mixed
a=msid-semantic: WMS
m=video 9 UDP/TLS/RTP/SAVPF 98
c=IN IP4 0.0.0.0
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=recvonly
a=rtcp:9 IN IP4 0.0.0.0
a=rtcp-rsize
a=rtcp-mux
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=setup:actpass
a=ice-ufrag:iIYk
a=ice-pwd:MM67D85LOn4AGaJqrZoJHJlT
a=candidate:37153457 1 udp 2113937151 633dd8b6-dae3-487d-bec3-0e24144442f6.local 61230 typ host generation 0 network-cost 999
a=candidate:3216145886 1 udp 2113939711 bd134863-1992-4778-bd10-6562b4866557.local 49342 typ host generation 0 network-cost 999

and the server's response is

v=0
o=rtc 391121705 0 IN IP4 127.0.0.1
s=-
t=0 0
a=group:BUNDLE 0
a=group:LS 0
a=msid-semantic:WMS *
a=ice-options:ice2,trickle
a=fingerprint:sha-256 A2:73:8F:40:D0:64:77:8C:D2:56:61:4F:13:44:7A:3F:BC:79:0B:59:5F:72:06:89:E0:57:2D:B0:66:4B:A0:8F
m=video 7141 UDP/TLS/RTP/SAVPF 98
c=IN IP4 46.4.38.241
a=mid:0
a=extmap:1 urn:ietf:params:rtp-hdrext:toffset
a=extmap:2 http://www.webrtc.org/experiments/rtp-hdrext/abs-send-time
a=extmap:3 urn:3gpp:video-orientation
a=extmap:4 http://www.ietf.org/id/draft-holmer-rmcat-transport-wide-cc-extensions-01
a=extmap:5 http://www.webrtc.org/experiments/rtp-hdrext/playout-delay
a=extmap:6 http://www.webrtc.org/experiments/rtp-hdrext/video-content-type
a=extmap:7 http://www.webrtc.org/experiments/rtp-hdrext/video-timing
a=extmap:8 http://www.webrtc.org/experiments/rtp-hdrext/color-space
a=extmap:9 urn:ietf:params:rtp-hdrext:sdes:mid
a=extmap:10 urn:ietf:params:rtp-hdrext:sdes:rtp-stream-id
a=extmap:11 urn:ietf:params:rtp-hdrext:sdes:repaired-rtp-stream-id
a=sendonly
a=rtcp:9 IN IP4 0.0.0.0
a=rtcp-mux
a=rtpmap:98 VP9/90000
a=rtcp-fb:98 goog-remb
a=rtcp-fb:98 transport-cc
a=rtcp-fb:98 ccm fir
a=rtcp-fb:98 nack
a=rtcp-fb:98 nack pli
a=fmtp:98 profile-id=0
a=setup:active
a=ice-ufrag:fDod
a=ice-pwd:k0MMaKc0pxVr9rF8lgbO0/
a=candidate:3 1 UDP 2130705919 2a01:4f8:221:3081::2 7141 typ host
a=candidate:1 1 UDP 2122317823 46.4.38.241 7141 typ host
a=candidate:2 1 UDP 2122317567 172.19.0.1 7141 typ host
a=end-of-candidates

My questions are thus:

Upvotes: 0

Views: 220

Answers (1)

oarfish
oarfish

Reputation: 4631

At least for this specific case, the answer turned out to be that I was not adding an SSRC on the sender (server) track in this specific code path.

The ssrc should be added to the track by whomever sends stream data as far as I understand, regardless of whether they offer or answer.

Not sure how I would be able to tell this from the SDP or the chrome webrtc logging.

Upvotes: 0

Related Questions