CODEWITHSUNDEEP
pythonpytorch
Dave Fol
Dave Fol

Reputation: 525

Why does an indexed tensor retain the original tensor's stride?

My understanding of stride is the number of steps you'd need to take to jump to the next part in an axis. So if you have a stride of (3,1) you'd need to take 3 steps to get to the next row and 1 step to get to the next column (assuming the first axis is row and the second axis is column).

Yet when I index into a pytorch tensor, b[1:, 1:] with shape (3,3) lopping off the first row and the first column, then query its stride, I get (3,1) instead of (2,1).

Why is this the case?

import unittest
import torch
class TestChapter3(unittest.TestCase):
    def setUp(self):
        self.a = torch.tensor(list(range(9)))
        self.b = self.a.view(3,3)
        self.c = self.b[1:, 1:]

    def test_index_of_view(self):
        print(self.c)
        self.assertEqual(self.c.size(), torch.Size([2, 2]))
        self.assertEqual(self.c.storage_offset(), 4)
        self.assertEqual(self.c.stride(), (2, 1)) # self.c.stride() is actually (3,1)


if __name__ == "__main__":
    unittest.main()

Upvotes: 0

Views: 98

Answers (1)

Marius
Marius

Reputation: 388

The self.c tensor still uses the same underlaying data storage as self.a and self.b. They are all just different views on the same storage.

self.c.storage_offset() tells you where the view of self.c begins (passing the first entire row and the first element of the second row = 4 elements in total) but the data is still stored in memory as one long array.

Since the underlying storage is still the same, the memory address to go one row down is still 3*sizeof(float) bytes or three elements further, which is what self.c.stride()[0] says.

In your example we have:

self.a = (visible=[0, 1, 2, 3, 4, 5, 6, 7, 8], data=[0, 1, 2, 3, 4, 5, 6, 7, 8])
self.b = (visible=[
    [0, 1, 2], 
    [3, 4, 5], 
    [6, 7, 8]
], data=[0, 1, 2, 3, 4, 5, 6, 7, 8])
self.c = (visible=[

       [4, 5], 
       [7, 8]
], data=[4, 5, 6, 7, 8])

And yes, the 6 in the last data array is not a mistake. The last data array is just an offset view of the same memory that self.a and self.b are referencing.

Upvotes: 1

Related Questions