How to plot graph on an image using matplotlib in python?

Question

I have integrated tensorflow object detection API with OpenCv to track a person within the frame so that the python script can draw a graph showing where the person entered into and exit the frame also where it moved inside the frame.

To draw graph I am using matplotlib. I have the x y coordinates and the graph is shown very nicely. Now I need to put an image in background of that graph. For this I have done below:

image = mpimg.imread(file)
plt.imshow(image)
plt.plot(x, y)
plt.plot(x[0], y[0], 'og')
plt.plot(x[-1], y[-1], 'ob')
plt.show()

In the above code, I am reading the file which is an image and then showing it on the graph. Doing this make my final graph not good looking as the image moves upward and the line graph remains on the lower portion while I wanted the line graph to be on the image. I referred to this question which is what I am trying to achieve but in the question which I have mentioned, the solution explains to draw a line over the image. In my scenario, I have to draw a graph which will have multiple coordinates.

I tried the solution explained in the referred question and used extent while showing the image in matplotlib, like below:

image = mpimg.imread(file)
plt.imshow(image, extent=[x[0], x[-1], y[0], y[-1]])
plt.plot(x, y)
plt.plot(x[0], y[0], 'og')
plt.plot(x[-1], y[-1], 'ob')
plt.show()

In the above code, I am using extent=[x[0], x[-1], y[0], y[-1]]) where x[0], x[-1], y[0], y[-1] means the first and last points of x and y coordinates, so that the image can fit properly on the graph. Doing this, I get below results:

As you can see the image is properly fitted in the graph but as soon as I plot my x y coordinates, it looks like below:

which is not as good as the line should be on the image. I am confused here as what I am doing wrong. I have tried to follow the same referred answer and used extent but while plotting the image moves a bit forward. Below are my coordinates:

x = [612, 590, 646, 712, 466, 475]
y = [623, 562, 557, 567, 530, 536]

Green dot resembles start and blue dot resembles end. Below is the image of the graph if I don't use a background image. One thing to note here is that x,y coordinate does not match with the above images but only matches with the below graph (without background image):

Can anyone please tell me what I am doing wrong here. Please help. Thanks

Answer 1

If the extent only uses the start and stop positions, the corners of the image will match to those positions. This will put any intermediate x and y values that are higher or lower outside the image.

I'm not sure what the x and y coordinates all include, but if they have points across the entire desired range, you could use the min and max values to get the extent.

image = mpimg.imread(file)
plt.imshow(image, extent=[min(x), max(x), min(y), max(y)])
plt.plot(x, y)
plt.plot(x[0], y[0], 'og')
plt.plot(x[-1], y[-1], 'ob')
plt.show()

However, I don't think this will be the case given these are tracked positions of the bounding box, in which case the extent shouldn't be based on those values at all. Instead, use the original image size, which I'm guessing is 1280x720.

plt.imshow(image, extent=[0, 1280, 0, 720])

It does appear to line up if you're tracking the mid-top of the bounding box.