Reputation: 21
Why can't I duplicate an MJPEG stream from a webcam to a V4L2 Loopback Device on Linux?
I'm trying to duplicate a USB webcam device V4L2 stream (/dev/video0) to a V4L2 Loopback Device (/dev/video99) at the highest resolution and framerate possible with the hardware available on a Raspberry Pi 4 running the latest Raspbian (also the minimum CPU load). I'm using FFMPEG version 7:4.1.6-1~deb10u1+rpt2 and v4l2loopback version 0.12.5.1.
The output from:
ffmpeg -f v4l2 -list_formats all -i /dev/video0 gives me the following:
[video4linux2,v4l2 @ 0xc1e1c0] Compressed: mjpeg : Motion-JPEG : 1600x1200 3264x2448 2592x1944 2048x1536 1280x960 1024x768 800x600 640x480 320x240 1600x1200
[video4linux2,v4l2 @ 0xc1e1c0] Raw : yuyv422 : YUYV 4:2:2 : 1600x1200 3264x2448 2592x1944 2048x1536 1280x960 1024x768 800x600 640x480 320x240 1600x1200
3264x2448px is the native CCD resolution, and the image is being used for machine vision purposes, so I'd like to use this maximum resolution at the highest framerate I can. The reason for the loopback device is so that I can view/stream the videofeed (with GStreamer) and take high resolution PNG snapshots simultaneously, using the command ffmpeg -f v4l2 -video_size 3264x2448 -i /dev/video99 -frames 1 capture.png -y From my understanding this means there is an MJPEG stream at this resolution and according to the command v4l2-ctl -d /dev/video0 --list-formats-ext I can manage 15fps:
ioctl: VIDIOC_ENUM_FMT
Type: Video Capture
[0]: 'MJPG' (Motion-JPEG, compressed)
Size: Discrete 1600x1200
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 3264x2448
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 2592x1944
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 2048x1536
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1280x960
Interval: Discrete 0.067s (15.000 fps)
Size: Discrete 1024x768
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1600x1200
Interval: Discrete 0.067s (15.000 fps)
Interval: Discrete 0.067s (15.000 fps)
[1]: 'YUYV' (YUYV 4:2:2)
Size: Discrete 1600x1200
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 3264x2448
Interval: Discrete 0.500s (2.000 fps)
Size: Discrete 2592x1944
Interval: Discrete 0.333s (3.000 fps)
Size: Discrete 2048x1536
Interval: Discrete 0.333s (3.000 fps)
Size: Discrete 1280x960
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 1024x768
Interval: Discrete 0.100s (10.000 fps)
Size: Discrete 800x600
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 640x480
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 320x240
Interval: Discrete 0.033s (30.000 fps)
Size: Discrete 1600x1200
Interval: Discrete 0.100s (10.000 fps)
Interval: Discrete 0.100s (10.000 fps)
I feel like I've tried every possible version of GStreamer/FFMPEG command to achieve this and have had varying success, but more often than not I come up against "av_interleaved_write_frame(): Cannot allocate memory" or some issue with converting M-JPEG to rawvideo for the loopback device...
What command should I be using? I'm at the end of my tether!
I've tried:
gst-launch-1.0 v4l2src device=/dev/video0 ! "image/jpeg,width=3264,height=2448,framerate=15/1" ! avdec_mjpeg ! "video/x-raw,format=YUY2,width=3264,height=2448,framerate=15/1" ! v4l2sink device=/dev/video99
and
ffmpeg -f v4l2 -video_size 3264x2448 -i /dev/video0 -vcodec rawvideo -pix_fmt yuyv422 -r 1 -f v4l2 /dev/video99
(EDIT) gives:
ffmpeg version 4.1.6-1~deb10u1+rpt2 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1~deb10u1+rpt2' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-vout-drm --enable-v4l2-request --enable-libudev --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 14226.770484, bitrate: 255688 kb/s
Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 3264x2448, 255688 kb/s, 2 fps, 2 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, video4linux2,v4l2, to '/dev/video99':
Metadata:
encoder : Lavf58.20.100
Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 3264x2448, q=2-31, 127844 kb/s, 1 fps, 1 tbn, 1 tbc
Metadata:
encoder : Lavc58.35.100 rawvideo
av_interleaved_write_frame(): Cannot allocate memory
frame= 1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:01.00 bitrate=N/A speed=15.8x
video:15606kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
and
ffmpeg -f v4l2 -video_size 3264x2448 -i /dev/video0 -vcodec rawvideo -pix_fmt yuv420p -r 15 -f v4l2 /dev/video99
(EDIT) gives:
ffmpeg version 4.1.6-1~deb10u1+rpt2 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1~deb10u1+rpt2' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-vout-drm --enable-v4l2-request --enable-libudev --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 13704.103283, bitrate: 255688 kb/s
Stream #0:0: Video: rawvideo (YUY2 / 0x32595559), yuyv422, 3264x2448, 255688 kb/s, 2 fps, 2 tbr, 1000k tbn, 1000k tbc
Stream mapping:
Stream #0:0 -> #0:0 (rawvideo (native) -> rawvideo (native))
Press [q] to stop, [?] for help
Output #0, video4linux2,v4l2, to '/dev/video99':
Metadata:
encoder : Lavf58.20.100
Stream #0:0: Video: rawvideo (I420 / 0x30323449), yuv420p, 3264x2448, q=2-31, 1438248 kb/s, 15 fps, 15 tbn, 15 tbc
Metadata:
encoder : Lavc58.35.100 rawvideo
av_interleaved_write_frame(): Cannot allocate memory
Last message repeated 7 times
frame= 8 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.53 bitrate=N/A dup=7 drop=0 speed=1.37x
video:93636kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
Conversion failed!
and
ffmpeg -f v4l2 -input_format mjpeg -i /dev/video0 -vcodec copy -f v4l2 /dev/video99
(EDIT) gives:
ffmpeg version 4.1.6-1~deb10u1+rpt2 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 8 (Raspbian 8.3.0-6+rpi1)
configuration: --prefix=/usr --extra-version='1~deb10u1+rpt2' --toolchain=hardened --incdir=/usr/include/arm-linux-gnueabihf --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-omx-rpi --enable-mmal --enable-neon --enable-rpi --enable-vout-drm --enable-v4l2-request --enable-libudev --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared --libdir=/usr/lib/arm-linux-gnueabihf --cpu=arm1176jzf-s --arch=arm
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
[mjpeg @ 0xdb5240] EOI missing, emulating
Input #0, video4linux2,v4l2, from '/dev/video0':
Duration: N/A, start: 12511.425271, bitrate: N/A
Stream #0:0: Video: mjpeg, yuvj422p(pc, bt470bg/unknown/unknown), 3264x2448, 15 fps, 15 tbr, 1000k tbn, 1000k tbc
[video4linux2,v4l2 @ 0xdb7f10] V4L2 output device supports only a single raw video stream
Could not write header for output file #0 (incorrect codec parameters ?): Invalid argument
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Last message repeated 1 times
I'm not sure what to troubleshoot next - looking at available memory, there's no problem, so what am I doing wrong? This copy of the feed is the last piece of the puzzle...
Upvotes: 2
Views: 2000
Answers (1)
Reputation: 21
I don't have enough points to comment, so I'll write what I've got so far. I'm having a similar issue, but using an RPi3 and am trying to stream from a webcam at 1440p to two loopback devices. One will be used for live streaming and the other for computer vision.
ffmpeg
I've got it to work, but the stream is 6-12 FPS, with a speed of 0.2-0.4x (too slow for what I want). I don't know how to fix this. Note the below is for 1 stream, but can easily be duplicated for two streams.
ffmpeg -hide_banner -vcodec mjpeg -s 2560x1440 -i /dev/video0 -map 0:v -vcodec rawvideo -vf format=yuv420p -f v4l2 /dev/video2
hide_banner hides a lot of the text
-vcodec rawvideo is already assumed, so isn't necessary
-vf format=yuv420p is used to convert pixel fmts ot smth v4l2 supports. I think this is the issue, because pixels are being converted, when what I (we) actually want is to send the compressed mjpeg stream directly to the loopback device.
Note that I didn't specify -r as it seemed to increase performance.
gst
This gives me corrupted images while streaming, but is very fast. By corrupted, I mean the colours are funky and sometimes the image is "boxy".
gst-launch-1.0 v4l2src device=/dev/video0 ! tee name=t ! queue ! v4l2sink device=/dev/videoX t. ! queue ! v4l2sink device=/dev/videoY
Source: comment by Olivier in duplicate webcam stream in a virtual device created with v4l2loopback
Note: I wrote this from my phone and don't know how to set the formatting, so I apologise for that.
Upvotes: 0
Related Questions
- How to write/pipe to a virtual webcam created by V4L2loopback module?
- Unable to route webcam video to virtual video device on Linux (via OpenCV)
- how to record camera to file while encoding stream to v4l2-loopback device?
- duplicate webcam stream in a virtual device created with v4l2loopback
- Play MJPEG stream to v4l2loopback device
- Using v4l2loopback and GStreamer with MJPEG cameras
- How to use GStreamer to stream from IP RTMP Camera to v4l2loopback Camera?
- How to change mjpeg to yuyv422 from a webcam to a v4l2loopback?
- Gstreamer capture and store mjpeg from webcam
- Not able to generate stable video stream over v4l2loopback device with python