LabVIEW: IMAQ colorImage to Array

Question

I'm working on an application where an image should be read from LabVIEW (by a camera) and preprocessed in python function (the Image should not be read from a file path, meaning we should not save the image shown in the camera but it should be read directly in the python function and that's why I thought of sending its pixels values to the function), for this I'm using system exec.

as a first step I was trying to read an Image from a file path to check if I can have its pixels values as an array and send it to the command line. I used the IMAQ colorImage to array and to convert it to a string I used "Flatten to string function" (since there's no 32U to string function) but as shown in the picture I'm getting strange symbols. Ps: I used system exec because I don't have the license for the other python LabVIEW connectivity

Please if you know how can I have the array of the image pixels to send it as an argument to the python function help me, I will be so grateful!

The new block Diagram :

John · Accepted Answer

You can use the flatten to string node to send the image data to your python script but you should note the following things:

The image data will be converted from a U32 2D-array to unsigned bytes (values 0-255). Some of these values are outside the standard ASCII character set, so they will not render correctly (they look like weird symbols or blank spaces). This also means that it is difficult to pass them as arguments but we can write them to stdin.

The flatten to string node (by default) appends the array size as I32s to the beginning of the data (which is useful) and uses the big-endian format (which is not).

The up-shot of this is that we need to override the default behaviour of the flatten to string node to use the little-endian format and strip off the first 8-bytes to convert to the array width and height in python.

LabVIEW stores image pixels in ARGB order but as we have flattened using the little-endian option, we end up with data in BGRA order which is conviently the expected channel order for openCV.

Putting this together and making sure we pass the image data into the standard input input of the system exec.vi we have the following LabVIEW and python code (Python Version 3.10.4):

import sys
import numpy as np
import cv2

# read std in as byteArray
dataInBuffer = sys.stdin.buffer.read()

# first 8 bytes are two, 4-byte signed integer values image width and height
width = int.from_bytes( dataInBuffer[0:3], "little", signed=True )
height = int.from_bytes( dataInBuffer[4:7], "little", signed=True)

# the rest of the data is 4-channel pixel data in BGRA format
imageBuffer = np.frombuffer(dataInBuffer[8:], dtype=np.uint8)

image = imageBuffer.reshape(width,height,4)

cv2.imshow('sample image',image)

cv2.waitKey(0) # waits until a key is pressed
cv2.destroyAllWindows() # destroys the window showing image

In response to the comment from @Asky

Sending images back to LabVIEW is indeed possible using the sys.stdout.buffer.write function.

Assuming that we want to use the Unflatten from string node in LabVIEW and we still have 4-channel BGRA data then the code is quite straight forward. We write the output image width and height as 4-byte signed integers first and then the image data. If you no longer have 4-channel data you could pad the data in python with zeros or do the padding in LabVIEW

The python code for these two cases is

import sys
import numpy as np
import cv2

# read std in as byteArray
dataInBuffer = sys.stdin.buffer.read()

# first 8 bytes are two, 4-byte signed integer values image width and height
width = int.from_bytes( dataInBuffer[0:3], "little", signed=True )
height = int.from_bytes( dataInBuffer[4:7], "little", signed=True)

# the rest of the data is 4-channel pixel data in BGRA format
imageBuffer = np.frombuffer(dataInBuffer[8:], dtype=np.uint8)

image = imageBuffer.reshape(width,height,4)

# process image
image = cv2.putText(image, 'Greetings from Python', (20,20), cv2.FONT_HERSHEY_SIMPLEX, 
                   0.5, (255,0,0), 1, cv2.LINE_AA)

outputWidthBytes = width.to_bytes(4,'little',signed=True)
outputHeightBytes = height.to_bytes(4,'little',signed=True)

# write to standard out
sys.stdout.buffer.write(outputWidthBytes)
sys.stdout.buffer.write(outputHeightBytes)
sys.stdout.buffer.write(image)

# write out three channel example
bgrImage = cv2.cvtColor(image, cv2.COLOR_BGRA2BGR)

# write to standard out as 3D array (width x height x channels)
sys.stdout.buffer.write(outputWidthBytes)
sys.stdout.buffer.write(outputHeightBytes)
sys.stdout.buffer.write((3).to_bytes(4,'little',signed=True))
sys.stdout.buffer.write(bgrImage)

# use flush to ensure all bytes are written to stdout before the program exits
sys.stdout.buffer.flush()

And the LabVIEW code extended to process the stdout data becomes

LabVIEW: IMAQ colorImage to Array

Answers (2)

Related Questions