How to convert raw YUV image to jpg

Question

I have a raw image that was taken with v4l2-ctl after the camera had been setup like:

# media-ctl -d /dev/media0 -l "'rzg2l_csi2 10830400.csi2':1 -> 'CRU output':0 [1]"
# media-ctl -d /dev/media0 -V "'rzg2l_csi2 10830400.csi2':1 [fmt:UYVY8_2X8/1280x960 field:none]"
# media-ctl -d /dev/media0 -V "'ov5645 0-003c':0 [fmt:UYVY8_2X8/1280x960 field:none]"

and then the picture got snapped with:

# v4l2-ctl --device /dev/video0 --stream-mmap --stream-to=frame.raw --stream-count=1

now I've tried multiple methods to convert this into a jpeg but nothing seems to yield the expected output

the raw file can be downloaded here: https://drive.google.com/file/d/1VqXnrJDYbzdtSsWfTlm2mX9rl1-Rl_7F/view?usp=sharing

I tried out the following command:

convert -verbose -size 1280x960 UYVY:frame.raw frame.bmp

which I found on Converting from YUV(UYVY) to RGB using imagemagick

but it doesn't do the trick

Mark Setchell · Accepted Answer

Your frame is 2457600 bytes and your pixel dimensions are 1280x960, so you have:

bits per pixel = 2457600 * 8 / (1280 * 960) = 16

You can get a list of the pixel formats that ffmpeg supports using:

ffmpeg -pix_fmts 2> /dev/null

Sample Output

FLAGS NAME            NB_COMPONENTS BITS_PER_PIXEL
-----
IO... yuv420p                3            12
IO... yuyv422                3            16
IO... rgb24                  3            24
IO... bgr24                  3            24
IO... yuv422p                3            16
IO... yuv444p                3            24
IO... yuv410p                3             9
...
...

That means you can get a list of pixel formats that contain Y, U and V with 16 bits per pixel like this:

ffmpeg -pix_fmts 2> /dev/null | awk '/y/ && /u/ && /16$/ {print}'  

IO... yuyv422                3            16
IO... yuv422p                3            16
IO... yuvj422p               3            16
IO... uyvy422                3            16
IO... yuv440p                3            16
IO... yuvj440p               3            16
IO... yvyu422                3            16

Now you can run a loop, iterating over all the 16-bit per pixel YUV formats and see what ffmpeg makes of your image - naming each result after the format so you can identify which is which:

ffmpeg -pix_fmts 2> /dev/null | 
   awk '/y/ && /u/ && /16$/ {print $2}' | 
      while read f; do 
         ffmpeg -y -s:v 1280x960 -pix_fmt $f -i frame.raw $f.jpg
      done

That gives you these files:

-rw-r--r--  1 mark  staff  304916  3 Feb 09:38 yuv440p.jpg
-rw-r--r--  1 mark  staff  227123  3 Feb 09:38 yuvj422p.jpg
-rw-r--r--  1 mark  staff   39543  3 Feb 09:38 yuyv422.jpg
-rw-r--r--  1 mark  staff   39545  3 Feb 09:38 yvyu422.jpg

And I guess that yuyv422.jpg is your image, so that means you can extract it with:

ffmpeg -y -s:v 1280x960 -pix_fmt yuyv422 -i frame.raw result.jpg

If you wanted to do that with ImageMagick, you could do something like this:

#!/bin/bash

python3 <



If yo don't like/have Python, that part can be replaced with some basic C as follows:
#include 
#include 

// Split YUYV file called "frame.raw" into separate channels with filenames "Y", "U" and "V"
// Compile with: clang -O3 splitter.c -o splitter

int main(){

   FILE    *in, *Y, *U, *V;
   uint8_t buffer[4];
   size_t  bytesRead;

   // Open input file and 1 output file per channel
   in = fopen("frame.raw", "rb");   
   Y  = fopen("Y", "wb");   
   U  = fopen("U", "wb");   
   V  = fopen("V", "wb");   

   // read up to sizeof(buffer) bytes
   while ((bytesRead = fread(buffer, 1, sizeof(buffer), in)) > 0)
   {
      fputc(buffer[0], Y);
      fputc(buffer[1], U);
      fputc(buffer[2], Y);
      fputc(buffer[3], V);
   }
}


Having had so much fun doing ffmpeg, Python, and C versions, I thought I'd try just doing it in the shell - converting bytes to lines and so I could pick alternate lines instead of alternate bytes. This works the same as the above:
#!/bin/bash

# Build JPEG image from YUYV image with packed bytes in order YUYVYUYV...
# Use "xxd" to convert bytes into lines, then extract alternate lines - which is easier than extracting bytes

H=960
W=1280
INPUT="frame.raw"

# Take top byte of every uint16 and put into "Y.pgm"
xxd -c1 -p "$INPUT" | sed -n 'p;n' | xxd -r -p | magick -size ${W}x${H} -depth 8 gray:- Y.pgm

# Take bottom byte of every 2nd uint16, starting at the 1st, resize up to full width and put into "U.pgm"
xxd -c1 -p "$INPUT" | sed -n 'n;p' | sed -n 'p;n' | xxd -r -p | magick -size $((W/2))x${H} -depth 8 gray:- -resize ${W}x${H}\! U.pgm

# Take bottom byte of every 2nd uint16, starting at the 2nd, resize up to full width and put into "V.pgm"
xxd -c1 -p "$INPUT" | sed -n 'n;p' | sed -n 'n;p' | xxd -r -p | magick -size $((W/2))x${H} -depth 8 gray:- -resize ${W}x${H}\! V.pgm

# Load the 3 channels, combine and convert to JPEG
magick {Y,U,V}.pgm -set colorspace YUV -combine -colorspace sRGB result.jpg

# Remove litter
rm {Y,U,V}.pgm


As regards colour cast removal, as I said in the comments, the " normal" way, AFAIK, is to get the average colour of the image and invert its Hue then blend that "negated cast" back with the original image to offset the original colour cast. Here is a crude attempt - if anyone knows better please ping me!
Step 1: Get average colour cast
magick result.jpg -resize 1x1\! cast.png


Step 2: Invert the cast
magick cast.png -modulate 100,100,0 correction.png


Step 3: Blend the original with the correction and brighten maybe
magick result.jpg correction.png -define compose:args=50,50 -compose blend -composite -auto-level result.jpg

Here are the original and corrected versions:

Obviously you can change the percentages for different degrees of "correction".

How to convert raw YUV image to jpg

Answers (1)

Related Questions