CODEWITHSUNDEEP
pythonnumpypytorch
syltruong
syltruong

Reputation: 2723

PyTorch conversion between tensor and numpy array: the addition operation

I am following the 60-minute blitz on PyTorch but have a question about conversion of a numpy array to a tensor. Tutorial example here.

This piece of code:

import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
np.add(a, 1, out=a)
print(a)
print(b)

yields

[2. 2. 2. 2. 2.]

tensor([2., 2., 2., 2., 2.], dtype=torch.float64)

However

import numpy as np
a = np.ones(5)
b = torch.from_numpy(a)
a = a + 1 #the diff is here 
print(a)
print(b)

yields

[2. 2. 2. 2. 2.]

tensor([1., 1., 1., 1., 1.], dtype=torch.float64)

Why are the outputs different?

Upvotes: 4

Views: 3275

Answers (2)

Jatentaki
Jatentaki

Reputation: 13103

This actually has little to do with PyTorch. Compare

import numpy as np
a = np.ones(5)
b = a

followed by either

np.add(a, 1, out=a)
print(b)

or

a = a + 1
print(b)

There is a difference between np.add(a, 1, out=a) and a = a + 1. In the former you retain the same object (array) a with different values (2 instead of 1); in the latter you get a new array, which is bound to the same variable name a and has values of 2. However, the "original" a is discarded and unless something else (b) points to it, would be deallocated. In other words, the first operation is in-place and the latter out-of-place. Since b holds on to the array originally found at a, reassigning a + 1 to a does not affect the value of b. An alternative in-place mutation syntax would be

a[:] = a + 1
print(b)

Regarding PyTorch, it's very simple. from_numpy creates a tensor which aliases an actual object (array), so it is equivalent to the b = a line in my first snippet. The tensor will track the changes in the array named a at the point of calling, rather than the changes of what the name a points to.

Upvotes: 5

Statistic Dean
Statistic Dean

Reputation: 5280

My take on this :

When you do 

b = torch.from_numpy(a)

b and a points to the same place in memory. From the doc : 

Converting a torch Tensor to a numpy array and vice versa is a breeze. The torch Tensor and numpy array will share their underlying memory locations, and changing one will change the other.

When you do 

np.add(a, 1, out=a)

You modify a in place while when you do 

a = a+1

You create a new array (named a as well), but that new array doesn't share underlying memory locations with b, so you're not affecting b.

Upvotes: 1

Related Questions