CODEWITHSUNDEEP
pythonopencvpytorchpython-imaging-library
user13325222
user13325222

Reputation:

Why is PIL used so often with Pytorch?

I noticed that a lot of dataloaders use PIL to load and transform images, e.g. the dataset builders in torchvision.datasets.folder.

My question is: why use PIL? You would need to do an np.asarray operation before turning it into a tensor. OpenCV seems to load it directly as a numpy array, and is faster too.

One reason I can think of is because PIL has a rich transforms library, but I feel like several of those transforms can be quickly implemented.

Upvotes: 9

Views: 5228

Answers (3)

prosti
prosti

Reputation: 46409

One possible reason PIL turns out to be frequent is because there are lot of examples online using PIL.Image.open method:

%matplotlib inline
from PIL import Image
img = Image.open(r"img.jpg")
# do more ...

Turns out that we don't need to use PIL, if we open the image using matplotlib.

%matplotlib inline
import matplotlib.pyplot as plt
import matplotlib.image as mpimg
img = mpimg.imread(r"img.jpg")
_ = plt.imshow(img)

Jupyter notebooks frequent need is to show the image, and matplotlib.pyplot.imshow is used frequently for that.

Upvotes: 0

Szymon Maszke
Szymon Maszke

Reputation: 24814

There is a discussion about adding OpenCV as one of possible backends in torchvision PR.

In summary, some reasons provided:

  • OpenCV2 loads images in BGR format which would require wrapper class to handle changing to RGB internally or format of loaded images backend dependent
  • This in turn would lead to code duplication in functional transforms in torchvision many of which use PIL operations (as transformations to support multiple backends would be pretty convoluted)
  • OpenCV loads images as np.array, it's not really easier to do transformations on arrays
  • Different representation might lead to hard to catch by users bugs
  • PyTorch's modelzoo is dependent on RGB format as well and they would like to have it easily supported
  • Doesn't play well with Python's multiprocessing (but it's no-issue as it was an issue for Python 2)

To be honest I don't see much movement towards this idea as there exists albumentations which uses OpenCV and can be integrated with PyTorch rather smoothly.

A little off-topic, but one can choose faster backend via torchvision.set_image_backend to Intel's accimage. Also Pillow-SIMD can be used as a drop-in replacement for PIL (it is supposedly faster and recommended by fastai project).

When it comes to performance benchmarks they do not seem too reliable and it's not that easy to tell AFAIK.

Upvotes: 15

Louis Lac
Louis Lac

Reputation: 6426

There is some elements of answer here and here.

TL,DR: because of historical reasons, some benchmarks (never trust bencharcks, though) and because PIL is lighter and easier to instal than OpenCV.

Upvotes: 2

Related Questions