CODEWITHSUNDEEP
pytorchtorchvision
Rick De
Rick De

Reputation: 59

Does using an image transform significantly slow down training?

I see image transforms used quite often by many deep learning researchers. They seem to be treated as if they are free GPU or CPU cycles.

Example:

transformations = transforms.Compose([
        transforms.Resize(255),
        transforms.CenterCrop(224),
        transforms.ToTensor(),
        transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
    ])

train_set = datasets.ImageFolder(data_dir + "/train", transform = transformations)

In this specific case would it not be infinitely better to process the images upfront and save them out for future use in some other format? I see this sometimes but extremely rarely.

Or am I wrong, and transformers on a GPU are just so fast it's not worth the extra code or hassle?

Upvotes: 2

Views: 3109

Answers (1)

Jack Rickman
Jack Rickman

Reputation: 76

It really depends on how you set up the dataloader. Generally, the transforms are performed on the CPU, and then the transformed data is moved to the GPU. Pytorch dataloaders have a 'prefetch_factor' argument that allows them to pre-compute your data (with transforms) in parallel with the GPU computing the model. That being said, with fixed transforms like you have here, pre-computing the entire dataset and saving it prior to computing could also be a valid strategy.

Upvotes: 3

Related Questions