CODEWITHSUNDEEP
pythondeep-learningpytorchresnetimage-classification
Coding Rookie
Coding Rookie

Reputation: 9

How can I use 2 images as a training sample in PyTorch？

I just begin learning deep learning and my first homework is to finish an leaves-classification system based on convolutional neural networks.I built a resnet-34 model with the code on github to do it.However,my teacher told me that the basic training unit in his dataset is an image pair.I should use 2 images(photos of the same leaf under different light conditions) as the input,combining two 3-channel images into one 6-channel image,but I don't know how to input 2 images and combine them into 6 channels.How can I do that?Are there any functions?Should I modify the structure of the resnet network? enter image description here
this is my dataset,you can see every two images are about the same leaf.

Upvotes: 1

Views: 2171

Answers (1)

Shai
Shai

Reputation: 114976

You have several issues to tackle:

  1. You need a Dataset with a __getitem__ method that returns 2 images (and a label) instead of the basic ones that returns a single image and a label. You'll probably need to customize your own dataset.
  2. Make sure the augmentations you apply to your images are applied in the same manner to each pair.
  3. You need to modify ResNet-34 network to get as an input 2 images, instead of one. See, e.g., this answer how that can be done.
  4. You need to change the first convolution layer to have 6 input channels instead of 3.
  5. If you want to use pre-trained weights you will not be able to load the existing state_dict of ResNet34 because of changes #3 and #4 - you'll have to do it manually for the first time.

Upvotes: 2

Related Questions