CODEWITHSUNDEEP
pythonpytorch
yujuezhao
yujuezhao

Reputation: 1105

Pytorch custom dataset: ValueError: some of the strides of a given numpy array are negative

I wrote a custom pytorch dataset, but ran into an error thhat seems quite unintelligible.

My custom dataset, 

class data_from_xlsx(Dataset):
    def __init__(self, xlsx_fp, path_col, class_cols_list):
        self.xlsx_file = pd.read_excel(xlsx_fp)
        self.path_col = path_col
        self.class_cols_list = class_cols_list

    def __len__(self):
        return get_xlsx_length(self.xlsx_file)

    def __getitem__(self, index):
        file_path = cols_from_xlsx(self.xlsx_file, index, 1, self.path_col) 
        feature = load_nii_file(file_path) # get 3D volume (x, y, z) 
        feature = np.expand_dims(feature, axis=0) # add channel (c, x, y, z)
        label = cols_from_xlsx(self.xlsx_file, index, 1, self.class_cols_list) # get label
        return feature, label.astype(np.bool)


def main():
dataset = data_from_xlsx("train.xlsx", "file_path", ["pos", "neg"], transformations, aug=True)
    data_loader = DataLoader(dataset, batch_size=4, shuffle=True)

    for (f, l) in data_loader:
        print("f shape", f.shape)
        print("l shape", l.shape)

An error is reported when I ran main(),

 File "d:\pytorch\lib\site-packages\torch\utils\data\dataloader.py", line 346, in __next__
    data = self.dataset_fetcher.fetch(index)  # may raise StopIteration
  File "d:\pytorch\lib\site-packages\torch\utils\data\_utils\fetch.py", line 47, in fetch
    return self.collate_fn(data)
  File "d:\pytorch\lib\site-packages\torch\utils\data\_utils\collate.py", line 80, in default_collate
    return [default_collate(samples) for samples in transposed]
  File "d:\pytorch\lib\site-packages\torch\utils\data\_utils\collate.py", line 80, in <listcomp>
    return [default_collate(samples) for samples in transposed]
  File "d:\pytorch\lib\site-packages\torch\utils\data\_utils\collate.py", line 65, in default_collate
    return default_collate([torch.as_tensor(b) for b in batch])
  File "d:\pytorch\lib\site-packages\torch\utils\data\_utils\collate.py", line 65, in <listcomp>
    return default_collate([torch.as_tensor(b) for b in batch])
ValueError: some of the strides of a given numpy array are negative. This is currently not supported, but will be added in future release

The reported error does't make sense to me, so I googled it. At first I thought I didn't change the feature from numpy.array to tensor, so I tried feature = torch.from_array(feature.copy()) and also tried transforms.TOTensor() but both attempts failed.

Upvotes: 3

Views: 8297

Answers (2)

GoingMyWay
GoingMyWay

Reputation: 17468

In some cases, you may need numpy.ascontiguousarray to return a contiguous array (ndim >= 1) in memory (C order). And then, you can use torch.from_numpy().

Click this link for more information.

Good luck.

Upvotes: 0

yujuezhao
yujuezhao

Reputation: 1105

Thanks to the advice from @jodag and @UsmanAli, I sovled this by return torch.from_numpy(feature.copy()) and torch.tensor(label.astype(np.bool)) So the whole thing should be,

class data_from_xlsx(Dataset):
    def __init__(self, xlsx_fp, path_col, class_cols_list):
        self.xlsx_file = pd.read_excel(xlsx_fp)
        self.path_col = path_col
        self.class_cols_list = class_cols_list

    def __len__(self):
        return get_xlsx_length(self.xlsx_file)

    def __getitem__(self, index):
        file_path = cols_from_xlsx(self.xlsx_file, index, 1, self.path_col) 
        feature = load_nii_file(file_path) # get 3D volume (x, y, z) 
        feature = np.expand_dims(feature, axis=0) # add channel (c, x, y, z)
        label = cols_from_xlsx(self.xlsx_file, index, 1, self.class_cols_list) # get label
        return torch.from_numpy(feature.copy()), torch.tensor(label.astype(np.bool))

Upvotes: 4

Related Questions