CODEWITHSUNDEEP
pytorch
Kermit
Kermit

Reputation: 5992

Is it safe to always use torch.tensor or torch.FloatTensor? Or do I need to treat Ints with care?

https://pytorch.org/docs/stable/tensors.html

I am trying to understand the difference between tensor, FloatTensor, IntTensor - and am wondering if I can just always stick to tensor... or maybe FloatTensor.

I am going to be using a mix of different tensors that will be:

{integers:labels, floats:continuous, one-hot-encoded:categoricals}

Do I need to explicitly set each of these as kinds of variables as different types of tensors? Will they all work as floats? Will they work in combination with each other?

Would this get me into trouble downstream?

l_o_l = [[1,2,3],[1,2,3],[1,2,3]]

int_tnz = th.FloatTensor(l_o_l)
int_tnz

tensor([[1., 2., 3.],
        [1., 2., 3.],
        [1., 2., 3.]])

>>> int_tnz.dtype
torch.float32


l_o_fl = [[1.1,2.2,3.3],[1.1,2.2,3.3],[1.1,2.2,3.3]]

int_tnz = th.tensor(l_o_fl)
int_tnz

tensor([[1.1000, 2.2000, 3.3000],
        [1.1000, 2.2000, 3.3000],
        [1.1000, 2.2000, 3.3000]])

>>> int_tnz.dtype

torch.float32

Upvotes: 4

Views: 15617

Answers (2)

Kermit
Kermit

Reputation: 5992

I've been able to use FloatTensor on all kinds of dtypes.

Can't average integer tensors, which makes sense.

a = th.tensor([1,1])
b = th.tensor([2,2])

th.mean(th.stack([a,b]))

RuntimeError: Can only calculate the mean of floating types. Got Long instead.

Upvotes: 0

Mohammad Arvan
Mohammad Arvan

Reputation: 623

CrossEntropyLoss (or NLLLoss) expect the target type to be Long. For example, the code below raises a RuntimeError:

import torch.nn 

criterion = torch.nn.CrossEntropyLoss()

predicted = torch.rand(10, 10, dtype=torch.float)
target = torch.rand(10) #.to(torch.long)
criterion(predicted, target)
# RuntimeError: expected scalar type Long but found Float

You need to uncomment the conversion to make it work. I cannot think of a bigger issue, but why bother converting the integers to float in the first place?

Regarding the use of torch.tensor and torch.FloatTensor, I prefer the former. torch.FloatTensor seems to be the legacy constructor, and it does not accept device as an argument. Again, I do not think this a big concern, but still, using torch.tensor increases the readability of the code.

Upvotes: 3

Related Questions