CODEWITHSUNDEEP
video-streaminggstreamerh.264rtp
szymeo
szymeo

Reputation: 431

how to extract H264 video from RTP packets according to RFC 3984?

I need to build receiver side of my udp video stream package to display video from my gstreamer udpsink in the browser and the only piece that is missing is extraction of h264 video from my rtp packets. I am looking for way to extract h264 from rtp packet on the fly (input - chunk from udp stream, output h264 data). Code receiving udp packets looks like this:

const dgram = require('dgram');

// Create a UDP server
const udp_server = dgram.createSocket('udp4');

// Bind the server to a port
udp_server.bind(5000);

// Handle incoming messages
udp_server.on('message', (msg, rinfo) => {
  /* handle rtp message containing h264 video */
});

I've tried with NodeJS rtp-parser without success. The only successful way I've managed to display a video stream is gstreamer command gst-launch-1.0 udpsrc port=5000 ! application/x-rtp,media=video,clock-rate=90000,encoding-name=H264,payload=96 ! rtph264depay ! avdec_h264 ! autovideosink. I am trying to reproduce rtph264depay step of this pipeline

Upvotes: 1

Views: 1060

Answers (1)

SolessChong
SolessChong

Reputation: 3407

You can use the rtp-parser package along with some custom logic. First, install the package:

const dgram = require('dgram');
const RtpParser = require('rtp-parser');

// Create a UDP server
const udp_server = dgram.createSocket('udp4');

// Bind the server to a port
udp_server.bind(5000);

// Initialize an empty buffer to store H.264 NAL units
let h264Buffer = Buffer.alloc(0);

// Handle incoming messages
udp_server.on('message', (msg, rinfo) => {
  const parsedRtpPacket = RtpParser.parseRtpPacket(msg);

  if (parsedRtpPacket.payloadType === 96) { // Assuming H.264 video has a payload type of 96
    const rtpPayload = parsedRtpPacket.payload;

    // H.264 NAL unit starts with the 0x000001 or 0x00000001 prefix
    const nalPrefix = Buffer.from([0x00, 0x00, 0x00, 0x01]);

    // Concatenate the NAL prefix and RTP payload to form the H.264 data
    const nalUnit = Buffer.concat([nalPrefix, rtpPayload]);

    // Append the NAL unit to the h264Buffer
    h264Buffer = Buffer.concat([h264Buffer, nalUnit]);
  }
});

// Get the H.264 video data
function getH264Data() {
  return h264Buffer;
}

Only basic example but could be a good start.

Upvotes: 1

Related Questions