CODEWITHSUNDEEP
javascriptffmpegraspberry-pivideo-streamingh.264
user19691992
user19691992

Reputation:

Decoding H.264 video from h264_v4l2m2m on Raspberry Pi 4

Problem Summary

I am trying to live-stream video from my Raspberry Pi 4 using the h264_v4l2m2m codec (HWA). At the receiving end, I am using the Broadway decoder. For lower resolutions (320x320 @ 60fps) the decoded video quality is fine. However, when I increase the resolution I start to notice a strange tessellation effect as in the following captures:

Tessellation effect on higher resolution (960x960)

I tried other codecs and players (mpeg1video and jsmpeg as a decoder), but at higher resolutions only the hardware accelerated H.264 reaches an acceptable frame rate. For my particular application I do not need audio.

Things tried

I am running the following command to launch ffmpeg on a separate process:

ffmpeg -hide_banner -fflags nobuffer -f rawvideo -pixel_format yuv420p -s 960x960  \
-i tcp://127.0.0.1:15000?listen=1 -f h264 -codec:v h264_v4l2m2m -s 960x960 -profile:v 66 \
-level 4.2 -coder 0 -bf 0 -an -b:v 4M tcp://0.0.0.0:15001?listen=1

I then send the H.264 NAL Units using a WebSocket.

The following snippet is my adjusted example application for decoding the received frames.

<!DOCTYPE html>
<html>

<head>
  <meta charset='utf-8'>
  <title>PiCamera H264 Streaming</title>
</head>

<body>
  <h1>PiCamera H264 Streaming</h1>
  <div id='viewer'></div>
  <script src='Decoder.js'></script>
  <script src='YUVCanvas.js'></script>
  <script src='Player.js'></script>
  <script>
    // player
    window.player = new Player({ useWorker: true, webgl: false })
    var playerElement = document.getElementById('viewer')
    playerElement.appendChild(window.player.canvas)
    // Websocket
    var wsUri = window.location.protocol.replace(/http/, 'ws') + '//' + window.location.hostname + ':20000'
    var ws = new WebSocket(wsUri)
    ws.binaryType = 'arraybuffer'
    ws.onopen = function (e) {
      console.log('Client connected')
      ws.onmessage = function (msg) {
        // decode stream
        let data = new Uint8Array(msg.data);
        console.log(data);
        window.player.decode(data);
      }
    }
    ws.onclose = function (e) {
      console.log('Client disconnected')
    }
  </script>
</body>

</html>

Question

What am I missing? Is it some option on ffmpeg, or could it be, that the decoder is not able to fully process the received frames. I have tried multiple bitrate settings to no avail.

Upvotes: 1

Views: 2784

Answers (1)

user19691992
user19691992

Reputation:

In the end I found jmuxer which can play live H.264 video without any problem. I got the hint from this post.

Upvotes: 1

Related Questions