Displaying PIL Images in Dash/Plotly

Question

I've been developing my Dash web application and am now looking into hosting it on my VM.

After I set up my environment, I'm unable to directly load PIL Image objects in html.Img elements. As they are rendered, an error will pop up and notify me that my PIL Image is not serializable.

This strikes me as weird, and possibly not an plotly error, but I have the exact same code, libraries and images causing error on my VM but running smoothly on my workstation.

After loading and doing some preprocessing, my Image object is passed to the html component as shown:

grid_main_images = <PIL.Image.Image image mode=RGB size=482x542 at 0x7FE88C04CD90>

html.Img(src=grid_main_imgs)

Again, the serialization error only occurs on my VM but not on my local machine. And here is the full error / traceback

Traceback (most recent call last):
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/dash/dash.py", line 1227, in add_context
    cls=plotly.utils.PlotlyJSONEncoder
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/_plotly_utils/utils.py", line 49, in encode
    encoded_o = super(PlotlyJSONEncoder, self).encode(o)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/json/encoder.py", line 199, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/json/encoder.py", line 257, in iterencode
    return _iterencode(o, 0)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/_plotly_utils/utils.py", line 119, in default
    return _json.JSONEncoder.default(self, obj)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Image is not JSON serializable

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 2463, in __call__
    return self.wsgi_app(environ, start_response)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 2449, in wsgi_app
    response = self.handle_exception(e)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 1866, in handle_exception
    reraise(exc_type, exc_value, tb)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/_compat.py", line 39, in reraise
    raise value
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/dash/dash.py", line 1291, in dispatch
    response.set_data(self.callback_map[output]['callback'](*args))
  File "/home/aegilsson/anaconda3/envs/diamond/lib/python3.7/site-packages/dash/dash.py", line 1242, in add_context
    ).replace('    ', ''))
dash.exceptions.InvalidCallbackReturnValue: 
The callback for property `children`
of component `tabs-content` returned a value
which is not JSON serializable.

In general, Dash properties can only be
dash components, strings, dictionaries, numbers, None,
or lists of those.

Answer 1

Not sure when it was introduced but both plotly.express.imshow() and plotly.graph_objects.Figure().add_layout_image() do accept PIL image out of the box:

In fact, if you provide any incompatible input like np.array to fig.add_layout_image({"source":np.array(my_pil_img)}) you will get the following ValueError:

The 'source' property is an image URI that may be specified as:

• A remote image URI string (e.g. 'http://www.somewhere.com/image.png')
• A data URI image string (e.g. 'data:image/png;base64,iVBORw0KGgoAAAANSU')
• A PIL.Image.Image object which will be immediately converted to a data URI image string See http://pillow.readthedocs.io/en/latest/reference/Image.html

Example:

from PIL import Image
#import pathlib

mysize = (512,512)

# if you have a real img
#path_to_file = pathlib.Path().cwd()/'dummy.png'
#img = Image.open(path_to_file)

# use dummy img here for example:
img = Image.new('RGBA', size=(1024, 1024), color=(155, 0, 0))


# I encountered issues with very big images, so best to make an in-place thumbnail
img = img.thumbnail(mysize, Image.ANTIALIAS)

Now for plotly.express:

import plotly.express as px
px.imshow(img)

or a bit more elaborate for plotly.graph_objects:

from plotly import graph_objects as go
fig = go.Figure()

# based on https://plotly.com/python/images/#zoom-on-static-images
# Constants
img_width, img_height = img.size
scale_factor = 1

# Add invisible scatter trace.
# This trace is added to help the autoresize logic work.
fig.add_trace(
    go.Scatter(
        x=[0, img_width * scale_factor],
        y=[0, img_height * scale_factor],
        mode="markers",
        marker_opacity=0
    )
)

# Configure axes
fig.update_xaxes(
    visible=False,
    range=[0, img_width * scale_factor]
)

fig.update_yaxes(
    visible=False,
    range=[0, img_height * scale_factor],
    # the scaleanchor attribute ensures that the aspect ratio stays constant
    scaleanchor="x"
)

# Add image
fig.add_layout_image(
    dict(
        x=0,
        sizex=img_width * scale_factor,
        y=img_height * scale_factor,
        sizey=img_height * scale_factor,
        xref="x",
        yref="y",
        opacity=1.0,
        layer="below",
        sizing="stretch",
        source=img)
)

# Configure other layout
fig.update_layout(
    width=img_width * scale_factor,
    height=img_height * scale_factor,
    margin={"l": 0, "r": 0, "t": 0, "b": 0},
)

# Disable the autosize on double click because it adds unwanted margins around the image
# More detail: https://plotly.com/python/configuration-options/
fig.show(config={'doubleClick': 'reset'})

Answer 2

You need to base64 encode the image and add some HTML headers.

def pil_to_b64(im, enc_format="png", **kwargs):
    """
    Converts a PIL Image into base64 string for HTML displaying
    :param im: PIL Image object
    :param enc_format: The image format for displaying. If saved the image will have that extension.
    :return: base64 encoding
    """

    buff = BytesIO()
    im.save(buff, format=enc_format, **kwargs)
    encoded = base64.b64encode(buff.getvalue()).decode("utf-8")

    return encoded

html.Img(id="my-img",className="image", src="data:image/png;base64, " + pil_to_b64(pil_img))

Credit: @Atli's comment pointed me in the right direction.