CODEWITHSUNDEEP
pythonmatplotlibjupyter
sutartmelson
sutartmelson

Reputation: 142

How to add visual annotations to an image using python in a jupyter notebook?

The overall objective is to view medical images from some server, add visual annotations to the image (i.e., highlighting/circling a tumor in red), then upload those images back to the server with the annotation as some sort of metadata.

The viewing/uploading/storing metadata is essentially complete, but I have yet to find any packages that allow me to draw on the images within jupyter notebooks. This could either manipulate the pixels or essentially create a new image on top of the displayed image.

I'm currently displaying the image data using matplotlib.

I've seen some use of javascript to track mouse movement and keystrokes, but can't figure out how to use that to my advantage. I can use any package, so long as it works within jupyter.

Upvotes: 3

Views: 3082

Answers (2)

sutartmelson
sutartmelson

Reputation: 142

Thanks to the suggestions from @michael_j_ward, I found a solution. I looked through those discussions and tutorials as well as read the documentation for the matplotlib axes. This is what I came up with/altered

import matplotlib.pyplot as plt
from matplotlib.lines import Line2D
import numpy as np
import dicom


class Annotator(object):
    def __init__(self, axes):
        self.axes = axes

        self.xdata = []
        self.ydata = []

    def mouse_move(self, event):
        if not event.inaxes:
            return

        x, y = event.xdata, event.ydata

        self.xdata.append(x)
        self.ydata.append(y)
        line = Line2D(self.xdata,self.ydata)
        line.set_color('r')
        self.axes.add_line(line)

        plt.draw()

    def mouse_release(self, event):
        # Erase x and y data for new line
        self.xdata = []
        self.ydata = []

path = '../sample.dcm'

data = dicom.read_file(path)

img = data.pixel_array

fig, axes = plt.subplots()
axes.imshow(img[0])
plt.axis("off")
plt.gray()
annotator = Annotator(axes)
plt.connect('motion_notify_event', cursor.mouse_move)
plt.connect('button_release_event', cursor.mouse_release)

axes.plot()

plt.show()

screenshot

It allows me to open images and draw on them to highlight or annotate significant portions of the image.

Upvotes: 3

Kevin K.
Kevin K.

Reputation: 1397

Inside Jupyter notebook you can use:

%matplotlib inline

Upvotes: -2

Related Questions