Reputation: 11
Problem description
Application implements Spring webflux websocket Netty server that forwards ffmpeg stream output to websocket protocol. This application allows multiple viewers to see the stream from their web browser by using jsmpeg javascript library.
This solution works as expected in Firefox web browser.
https://i.sstatic.net/X4Fm4.png
But in Chrome web browser the stream fails no load. Chrome browser cannot get data from the websocket connection.
Follows an screenshot of the problem seen in Chrome DevTools.
https://i.sstatic.net/xWvll.png
Question
What should be changed in this solution so that stream can be opened in Chrome?
How to reproduce the problem
My research of the problem
Turned on TRACE logs and opened the stream in Chrome browser. I see following error in application logs:
2022-09-02 15:26:36.952 - DEBUG [reactor-http-nio-3] r.n.h.s.HttpServerOperations --- [08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] New http connection, requesting read
2022-09-02 15:26:36.952 - DEBUG [reactor-http-nio-3] r.n.t.TransportConfig --- [08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Initialized pipeline DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.left.httpTrafficHandler = reactor.netty.http.server.HttpTrafficHandler), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.955 - DEBUG [reactor-http-nio-3] r.n.h.s.HttpServerOperations --- [08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Increasing pending responses, now 1
2022-09-02 15:26:36.955 - DEBUG [reactor-http-nio-3] r.n.h.s.HttpServer --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Handler is being applied: org.springframework.http.server.reactive.ReactorHttpHandlerAdapter@6c90595a
2022-09-02 15:26:36.956 - TRACE [reactor-http-nio-3] o.s.w.s.a.HttpWebHandlerAdapter --- [08cf17fd-2] HTTP GET "/stream", headers={masked}
2022-09-02 15:26:36.962 - DEBUG [reactor-http-nio-3] o.s.w.r.h.SimpleUrlHandlerMapping --- [08cf17fd-2] Mapped to com.edi.pacs.video_stream.FfmpegStreamSocketHandler@7ead1d80
2022-09-02 15:26:36.963 - DEBUG [reactor-http-nio-3] r.n.ReactorNetty --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Removed handler: reactor.left.httpTrafficHandler, pipeline: DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] r.n.ReactorNetty --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Non Removed handler: reactor.left.accessLogHandler, context: null, pipeline: DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] r.n.ReactorNetty --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Non Removed handler: reactor.left.httpMetricsHandler, context: null, pipeline: DefaultChannelPipeline{(reactor.left.httpCodec = io.netty.handler.codec.http.HttpServerCodec), (reactor.right.reactiveBridge = reactor.netty.channel.ChannelOperationsHandler)}
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] i.n.h.c.h.w.WebSocketServerHandshaker --- [id: 0x08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] WebSocket version V13 server handshake
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] i.n.h.c.h.w.WebSocketServerHandshaker --- WebSocket version 13 server handshake key: BCbwUez1uBWHEcnoIKUeBw==, response: 2L7i3HHoJx/Q9QKFH1IWl7ZYzYg=
2022-09-02 15:26:36.964 - DEBUG [reactor-http-nio-3] i.n.h.c.h.w.WebSocketServerHandshaker --- Requested subprotocol(s) not supported: null
2022-09-02 15:26:36.965 - DEBUG [reactor-http-nio-3] o.s.w.r.s.a.ReactorNettyWebSocketSession --- [08cf17fd-2] Session id "b528e63" for http://localhost:8080/stream
2022-09-02 15:26:36.966 - TRACE [reactor-http-nio-3] o.s.w.r.s.a.ReactorNettyWebSocketSession --- [08cf17fd-2] Sending WebSocket BINARY message (8 bytes)
2022-09-02 15:26:36.966 - TRACE [reactor-http-nio-3] i.n.h.c.h.w.WebSocket08FrameEncoder --- Encoding WebSocket Frame opCode=2 length=8
2022-09-02 15:26:36.966 - TRACE [reactor-http-nio-3] o.s.w.s.a.HttpWebHandlerAdapter --- [08cf17fd-2] Completed 200 OK, headers={}
2022-09-02 15:26:36.967 - TRACE [reactor-http-nio-3] o.s.h.s.r.ReactorHttpHandlerAdapter --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Handling completed
2022-09-02 15:26:36.967 - INFO [reactor-http-nio-3] r.F.S.2 --- onSubscribe(SinkManyBestEffort.DirectInner)
2022-09-02 15:26:36.967 - INFO [reactor-http-nio-3] r.F.S.2 --- request(128)
2022-09-02 15:26:36.967 - ERROR [reactor-http-nio-3] r.n.t.ServerTransport --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] onUncaughtException(ws{uri=/stream, connection=SimpleConnection{channel=[id: 0x08cf17fd, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815]}})
java.io.IOException: An established connection was aborted by the software in your host machine
at java.base/sun.nio.ch.SocketDispatcher.read0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:46)
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:389)
at io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:258)
at io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1132)
at io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:357)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:151)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:832)
2022-09-02 15:26:36.968 - TRACE [reactor-http-nio-3] r.n.c.ChannelOperations --- [08cf17fd-1, L:/[0:0:0:0:0:0:0:1]:8080 - R:/[0:0:0:0:0:0:0:1]:54815] Disposing ChannelOperation from a channel
java.lang.Exception: ChannelOperation dispose stack
at reactor.netty.channel.ChannelOperations.dispose(ChannelOperations.java:196)
at reactor.netty.transport.ServerTransport$ChildObserver.onUncaughtException(ServerTransport.java:467)
at reactor.netty.channel.FluxReceive.drainReceiver(FluxReceive.java:232)
at reactor.netty.channel.FluxReceive.onInboundError(FluxReceive.java:453)
at reactor.netty.channel.ChannelOperations.onInboundError(ChannelOperations.java:488)
at reactor.netty.channel.ChannelOperationsHandler.exceptionCaught(ChannelOperationsHandler.java:126)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
at io.netty.channel.AbstractChannelHandlerContext.fireExceptionCaught(AbstractChannelHandlerContext.java:273)
at io.netty.channel.DefaultChannelPipeline$HeadContext.exceptionCaught(DefaultChannelPipeline.java:1377)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:302)
at io.netty.channel.AbstractChannelHandlerContext.invokeExceptionCaught(AbstractChannelHandlerContext.java:281)
at io.netty.channel.DefaultChannelPipeline.fireExceptionCaught(DefaultChannelPipeline.java:907)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.handleReadException(AbstractNioByteChannel.java:125)
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:177)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658)
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:832)
2022-09-02 15:26:36.968 - INFO [reactor-http-nio-3] r.F.S.2 --- cancel()
My assumption is that Chrome doesn't "like" something about the given response from the Spring application and then Google closed the connection.
Next what I'll try is to use Jetty or Undertow as server but if it solves my problem, it won't solve my question - why doesn't it work with Netty.
Upvotes: 0
Views: 564
Reputation: 11
Fixed the problem. Problem was that some web browsers are "allergic" to subprotocol header being empty.
chrome empty subprotocol header network log
There are two possible solutions for this problem.
First solution, fill the subprotocol value. Done the fix in https://github.com/haraldsegliens/spring-webflux-websocket-jsmpeg-chrome-problem/tree/solution/use-protocol and Chrome now handles the stream correctly.
chrome now works by using subprotocol
Second solution, fix jsmpeg so that it doesn't write subprotocol header with empty value instead it should skip subprotocol header. Done fix in https://github.com/haraldsegliens/spring-webflux-websocket-jsmpeg-chrome-problem/tree/solution/fix-in-jsmpeg
Following pull request that fixes the problem in jsmpeg repository: https://github.com/phoboslab/jsmpeg/pull/143
chrome now works by fixing jsmpeg
Upvotes: 1