CODEWITHSUNDEEP
pythonmachine-learningcomputer-visiondatasetpytorch
dsillman2000
dsillman2000

Reputation: 1026

Trying to load a custom dataset in Pytorch

I'm just starting out with PyTorch and am, unfortunately, a bit confused when it comes to using my own training/testing image dataset for a custom algorithm. For starters, I am making a small "hello world"-esque convolutional shirt/sock/pants classifying network. I've only loaded a few images and am just making sure that PyTorch can load them and transform them down properly to 32x32 usable images. My ImageFolder is set up like so:

imgs/socks/sockimages.jpeg
imgs/pants/pantsimages.jpeg
imgs/shirt/shirtimages.jpeg

and a similar setup for my testing images folder. According to my current knowledge, the image loader built into PyTorch should read the labels from the subfolder names within the training/test images. However, I'm getting a TypeError complaining that my iterator is not iterable. Here's my code and the error:

import torch
import torchvision
import torchvision.datasets as dset
import torchvision.transforms as transforms

transform = transforms.Compose(
[transforms.ToTensor(),
 transforms.Scale((32,32)),
 transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

trainset = dset.ImageFolder(root="imgs",transform=transform)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=4,shuffle=True,         num_workers=2)

testset = dset.ImageFolder(root='tests',transform=transform)
testloader = torch.utils.data.DataLoader(testset, batch_size=4,shuffle=True,     num_workers=2)

classes=('shirt','pants','sock')

import matplotlib.pyplot as plt
import numpy as np

# functions to show an image
def imshow(img):
    img = img / 2 + 0.5     # unnormalize
    npimg = img.numpy()
    plt.imshow(np.transpose(npimg, (1, 2, 0)))

# get some random training images
dataiter = iter(trainloader)
images, labels = dataiter.next()

# show images
imshow(torchvision.utils.make_grid(images))
# print labels
print(' '.join('%5s' % classes[labels[j]] for j in range(4)))

Error:

TypeError: 'builtin_function_or_method' object is not iterable

It says that it is in reference to the line containing dataiter.next(), meaning that the compiler believes that I cannot iterate dataiter?

Please help! Thanks in advance,

-David Sillman, new to PyTorch

Upvotes: 4

Views: 10328

Answers (3)

Manuel Lagunas
Manuel Lagunas

Reputation: 2751

I think the error comes because in the transform.Compose you are doing first .ToTensor() and, instead, you should do .Scale(). Pytorch has transformations on tensors and on PIL Images, that are not interchangeable. Reading the docs it says

class torchvision.transforms.Scale(size, interpolation=2) [...] Rescale the input PIL.Image to the given size.

While you are changing that image to a Pytorch tensor before scaling thus making it crash.

It should be changed to:

transform = transforms.Compose(
                   [transforms.Scale((32,32)),
                    transforms.ToTensor(),
                    transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

You will get this error when applying PIL Image transformations on tensors.

Upvotes: 5

zouying
zouying

Reputation: 1617

For your question, I think transforms.ToTensor() before transform.Scale((32, 32)) is not right.

In the document of Scale::__call__(self, img) already show

Args: img(PIL.Image): Image to be scaled.

So the input for Scale is PIL.Image not Tensor.

transform = transforms.Compose(
[transforms.ToTensor(),
 transforms.Scale((32,32)),
 transforms.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5))])

So you could try the following one:

transform = transforms.Compose([transforms.Scale((32,32)),
                                transforms.ToTensor(),
                                transforms.Normalize((0.5, 0.5, 0.5), 
                                                     (0.5, 0.5, 0.5))])

You can the script to load your custom dataset at this gist. Click here to the result of the script.

I post the full classification in custom images, you could check it at github.com/xpzouying/animals-classification

Upvotes: 1

Achaiah
Achaiah

Reputation: 91

This might be as simple as you not providing the correct path to the "imgs" folder. Are you running your program from the same folder as the "imgs" folder? Try specifying an absolute path to your "imgs" folder and see if it helps.

Upvotes: 0

Related Questions