ModuleNotFoundError: No module named 'matplotlib' when using plt.imread() in embedded Python code

Question

I am trying to read an image using plt.imread() in embedded Python code within my DATA function. The code is as follows:

IMPORT PYTHON3 AS PYTHON;
IMPORT Std;
IMPORT Image;

myRec := RECORD
    STRING id;
    DATA image;
  END;

ds := DATASET('img.flat', myRec, THOR);

dims := GetImageDimensions(ds[2].image);

DATA ReadImage(DATA image, SET OF INTEGER dims) := EMBED(Python)
      import cv2
      import numpy as np
      import matplotlib.pyplot as plt
      image_np = np.frombuffer(image, dtype='uint8')
      dims = tuple(dims)
      image = plt.imread(io.BytesIO(image_np))
      print(type(image))
      np_array = np.array(image)
      print(np_array.shape)
      print(np_array)
      return bytearray(image_np)
    ENDEMBED;

OUTPUT(ReadImage(ds[2].image,dims));

However, I am getting the following error: Error, eclagent, 0, System error: 0: Graph graph1[1], remoteresult[4]: pyembed: No module named 'matplotlib', Master exception - caused by (0, pyembed: No module named 'matplotlib')

I have already installed the matplotlib module using pip install matplotlib==3.7.1 and checked that it is installed in my system. How can I resolve this error and use plt.imread() successfully within my embedded Python code?

Answer 1

First it's important to verify if the packages required are installed on the environment running the HPCC Systems platform. Having it installed only on your PC is not enough.

In the example below I used a Virtual Machine running the platform, but the same idea applies to bare metal clusters, cloud clusters, etc.

1. Login to the environment running the platform

After the VM has properly started, you can use the default credentials (hpccdemo) to login and install any packages you need:

You can also use any command prompt you find more friendly to SSH into the VM:

2. Installing the matplotlib on the environment

Now it's time to install all packages and libraries you need. For the matplotlib simply run the following command:

$ sudo apt-get install python3-matplotlib

Since we have "two pythons installed" on the machine, one being the standard python that comes with the OS and the second is the one you installed for using PIP and etc, with the previous command line we are making sure that we are using the right one (Python 3) to install it.

3. Test it

Finally, we are all set to use matplotlib. A simple way to verify if it was properly added to the platform is using a version check code:

IMPORT PYTHON3 AS PYTHON;
IMPORT Std;

STRING ReadImage() := EMBED(Python)
    import matplotlib
    return ('matplotlib: {}'.format(matplotlib.__version__))
ENDEMBED;

OUTPUT(ReadImage());

Looking at the result on the ECLWatch page, we'll see this: