CODEWITHSUNDEEP
androidwebrtcgstreamerturn
Jakub Licznerski
Jakub Licznerski

Reputation: 1088

WebRTC connection not working outside of local network

We have the following setup for webrtc two-way video and audio streaming:

Mobile

Android application using Google WebRTC implementation java wrapper. Tested these two libs:

implementation 'org.webrtc:google-webrtc:1.0.+'
implementation 'com.github.webrtc-sdk:android:92.4515.03' // https://github.com/webrtc-sdk/android

RPi

Gstreamer with WebRTC plugin based on gst-examples with a few modifications.

Browser

Also slightly modified gst-examples Javascript implementation. It is based on native browser WebRTC support.

TURN

Coturn server working in Docker (coturn/coturn).

The problem

When calling from RPi to browser or mobile to browser everything works fine both in local network and via Internet (through TURN server). However when trying to connect RPi and Android device via Internet it gets stuck (local network works as well). The devices communicate with each other, negotiate SDPs and ICE candidates and the TURN server opens the connections properly, but there is no audio/video.

When looking at Android logs, the first difference is that there is no ICE connection change triggered:

PeerConnectionObserver: onStandardizedIceConnectionChange: CHECKING

nor

PeerConnectionObserver: onConnectionChange: CONNECTING

which happens before ICE candidates exchange. After ICE candidates have been sent there is no further communication with the other device and no status changes are being fired. Like for example these in the working case:

PeerConnectionObserver: onSelectedCandidatePairChanged: org.webrtc.CandidatePairChangeEvent@a856333
PeerConnectionObserver: onStandardizedIceConnectionChange: CONNECTED
PeerConnectionObserver: onConnectionChange: CONNECTED
PeerConnectionObserver: onIceConnectionChange CONNECTED
[Internal data channel]: onStateChange
PeerConnectionObserver: onDataChannel org.webrtc.DataChannel@b76f6f0
[External data channel]: onStateChange

The coturn logs don't show anything suspicious at all.

Would appreciate any tips or help of any kind in further investigating this issue.

EDIT:

With @RSATom suggestion we've looked more closely to ICE and SDP messages being sent and found no discrapancies there. The SDPs are identical for mobile-RPi communication through Internet and locally. The only difference are ids generated randomly. The ICEs also do not differ much. It's identical for the first sender (offerer) and slightly different for answerer because it contains the TURN details:

via Internet

video0:0:candidate:228040959 1 udp 2122260223 10.111.215.156 53556 typ host generation 0 ufrag 3SNh network-id 3 network-cost 900::UNKNOWN  
video0:0:candidate:559267639 1 udp 2122202367 ::1 48566 typ host generation 0 ufrag 3SNh network-id 2::UNKNOWN      
video0:0:candidate:1510613869 1 udp 2122129151 127.0.0.1 41279 typ host generation 0 ufrag 3SNh network-id 1::UNKNOWN   
video0:0:candidate:1876313031 1 tcp 1518222591 ::1 46051 typ host tcptype passive generation 0 ufrag 3SNh network-id 2::UNKNOWN 
video0:0:candidate:344579997 1 tcp 1518149375 127.0.0.1 60259 typ host tcptype passive generation 0 ufrag 3SNh network-id 1::UNKNOWN    
video0:0:candidate:842163049 1 udp 1686052607 31.0.91.196 6742 typ srflx raddr 10.111.215.156 rport 53556 generation 0 ufrag 3SNh network-id 3 network-cost 900:stun:3.70.23.20:3478:UNKNOWN    
video0:0:candidate:593469510 1 udp 41885439 172.31.0.169 14307 typ relay raddr 31.0.91.196 rport 6742 generation 0 ufrag 3SNh network-id 3 network-cost 900:turn:3.70.23.20:3478?transport=udp:UNKNOWN

via local WiFi

video0:0:candidate:2858526953 1 udp 2122260223 192.168.0.21 38123 typ host generation 0 ufrag rwx/ network-id 3 network-cost 10::UNKNOWN
video0:0:candidate:559267639 1 udp 2122202367 ::1 52058 typ host generation 0 ufrag rwx/ network-id 2::UNKNOWN
video0:0:candidate:1510613869 1 udp 2122129151 127.0.0.1 39469 typ host generation 0 ufrag rwx/ network-id 1::UNKNOWN
video0:0:candidate:842163049 1 udp 1686052607 178.235.191.135 13607 typ srflx raddr 192.168.0.21 rport 38123 generation 0 ufrag rwx/ network-id 3 network-cost 10:stun:3.70.23.20:3478:UNKNOWN
video0:0:candidate:1876313031 1 tcp 1518222591 ::1 51551 typ host tcptype passive generation 0 ufrag rwx/ network-id 2::UNKNOWN
video0:0:candidate:344579997 1 tcp 1518149375 127.0.0.1 41007 typ host tcptype passive generation 0 ufrag rwx/ network-id 1::UNKNOWN
video0:0:candidate:593469510 1 udp 41885439 172.31.0.169 13744 typ relay raddr 178.235.191.135 rport 13607 generation 0 ufrag rwx/ network-id 3 network-cost 10:turn:3.70.23.20:3478?transport=udp:UNKNOWN

The order of ICE messages seems not to be persisted.

A diff of full logs (Internet left, local right): https://www.diffchecker.com/elEA6rkJ

ICE messages diff: https://www.diffchecker.com/C1TzPcMm

EDIT 2:

I missed that some of the logs were not present. Full log diff for connectivity via Internet (working vs not-working) available here. There are a few differences:

  1. Invalid connection misses a lot of logs informing of connection creation and keepalive and still displays this instead:
tag: basic_ice_controller.cc 
message: (line 541): Sorting 0 available connections
  1. For the invalid connection RTP SSRC transport_cc is turned off
  2. Gstreamer uses stream aliases (like video0 and audio0) which then cause ICE candidate errors like this:
ReadyToUseRemoteCandidate: Invalid candidate. Mid 0 specified but no media section with that mid found.
  1. There are failed TCP requests to TURN:
LS_INFO
tag: turn_port.cc 
message: (line 375): Port[31f33800:video0:1:0:relay:Net[lo:127.0.0.x/8:Loopback:id=1]]: Trying to connect to TURN server via tcp @ sicdev-turn.ddns.net:3478

LS_INFO
tag: android_network_monitor.cc 
message: (line 422): Find network handle.

LS_WARNING
tag: android_network_monitor.cc 
message: (line 307): BindSocketToNetwork unable to find network handle for addr: 127.0.0.x ifname: lo

LS_VERBOSE
tag: physical_socket_server.cc 
message: (line 207): Binding socket to loopback address failed; result: -3

LS_ERROR
tag: basic_packet_socket_factory.cc 
message: (line 186): TCP connect failed with error 22

LS_ERROR
tag: turn_port.cc 
message: (line 379): Failed to create TURN client socket

Upvotes: 2

Views: 4499

Answers (1)

RSATom
RSATom

Reputation: 867

What I would do in your case:

  1. Add logs for create-offer/create-answer/set-offer/set-answer requests
  2. Add logs for every incoming/outgoing Ice Candidate and end-of-candidates

Then look to logs and check:

  1. All Ice Candidates are really delivering to other party.
  2. end-of-candidates delivering after all other Ice Candidates.
  3. Ice Candidates coming after set-offer/set-answer
  4. Your STUN/TURN server is really accessible by both parties (with https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/)
  5. You have IPv6 configured on your STUN/TURN server (some GSM providers could not use IPv4 at all)

Upd 1

According to logs your Android device has IPv4. So there is no need to worry about IPv6 right now. About end-of-candidates - you can look how I generate it with webrtcbin here. The similar code for Android:

    private val connectionObserver = object: PeerConnection.Observer {
        // ... some overrides skipped ...

        override fun onIceCandidate(candidate: IceCandidate) {
            Log.d(TAG, "onIceCandidate \"$candidate\"")
        }
        override fun onIceGatheringChange(state: PeerConnection.IceGatheringState) {
            Log.d(TAG, "IceGathering state: $state")
            if(state == PeerConnection.IceGatheringState.COMPLETE) {
                Log.d(TAG, "onIceCandidate \"a=end-of-candidates\"")
            }
        }

        // ... some overrides skipped ...
    }

Also please check following:

On Android

  1. You are really call org.webrtc.PeerConnection.addIceCandidate for every incoming Ice Candidate
  2. You do it after org.webrtc.PeerConnection.setRemoteDescription call

On RPi

  1. You are really call g_signal_emit_by_name with add-ice-candidate for every incoming Ice Candidate

Upd 2

I've noticed your relay Ice Candidate points to private network IP (172.31.0.169). This can mean you have wrong config for your TURN server.

Upd 3

Please check with https://webrtc.github.io/samples/src/content/peerconnection/trickle-ice/ you get relay Ice Candidate with valid IP on both sides.

Upvotes: 4

Related Questions