stream video from pi cam to unity project

Question

I am using a raspberry pi and pi cam to stream video from my pi to my PC. I've gotten this to work using gstreamer on the pi and the PC, but now i want to use the stream in a Unity augmented reality project.

Here's the code running on the pi:

#!/bin/bash
clear
raspivid -n -t 0 -w 960 -h 720 -fps 30 -b 6000000 -o - |  gst-launch-1.0 -e -vvvv fdsrc ! h264parse ! rtph264pay pt=96 config-interval=5 ! udpsink host=***YOUR_PC_IP*** port=5000

In the Unity app i am using Vuforia for augmented reality. I want to use a pi for the video so that the webcam can be portable.

Any ideas on how to get the stream in Unity C# from the pi?

Answer 1

It's actually possible to stream Pi's camera to Unity in this way, with below solution in two parts.

Part 1. Raspberry Pi Setup: install StereoPi OS and run below commands.

//1. Connect to your StereoPi via ssh
ssh root@192.168.xx.xx
Pwd: root
 
//2. Stop Default Stream
/opt/StereoPi/stop.sh
 
//3. Start StereoPi stream, either in UDP or TCP
//where 192.168.1.10 3001 is the IP and port of your target device(receiver)
//command for UDP Stream
raspivid -t 0 -w 1280 -h 720 -fps 30 -3d sbs -cd MJPEG -o - | nc 192.168.1.10 3001 -u

//command for TCP Stream
raspivid -t 0 -w 1280 -h 720 -fps 30 -3d sbs -cd MJPEG -o - | nc 192.168.1.10 3001

Part 2. Unity Setup: import FMETP STREAM input your project, and add component "FMNetworkmanager.cs", Select network type as StereoPi. The detailed & the latest guide can refer to their documentation page.