What is the difference between a tensor and a multi-d matrix in Tensorflow?

Question

I am following the tensorflow CNN tutorial and bumped into the question of what programatically is the difference between a 'tensor' and a multi-dimensional matrix in Tensorflow and in general as well.

I tried to research on my own what a tensor is and what I have found out is: it it can be of order n, where every element hold information of n dimensions. E.g. if we have a tensor A and a data point with coordinates (3,2,5,4), then we are talking about a 4-D tensor A with one element. Is that correct?

Other articles that I found say that a tensor is the same as an array with the difference that a tensor's elements may transform. Again I don't see the difference betwen a tensor and a normal multi-dimensional array. We can always apply a function on the array and transform the elements.

Could you please try to clarify the definitions/properties and differences?

Answer 1

Slide 7 of this presentation has a nice visualization of various tensors.

https://www.slideshare.net/yokotatsuya/principal-component-analysis-for-tensor-analysis-and-eeg-classification

I wondered the same in the beginning. The answer is mundane though.

A "tensor" is the general purpose word given to an N-dimensional set of values. We have mathematical names for the low-rank tensors: scalars, vectors, matrices.

In tensorflow the rank of a tensor is its dimensionality. Here are some examples:

---------------------------------------------------------------
| Rank of  | Math     | Example                               |
| tensor   | entity   |                                       |
---------------------------------------------------------------
|    0     | Scalar   | x = 42                                |
|    1     | Vector   | z = [10, 15, 20]                      |
|    2     | Matrix   | a = [[1 0 2 3],                       |
|          |          |      [2 1 0 4],                       |
|          |          |      [0 2 1 1]]                       |
|    3     | 3-Tensor | A single image of shape:              |
|          |          | [height, width, color_channels]       |
|          |          | ex: [1080, 1920, 3]                   |
|    4     | 4-Tensor | A batch of images with shape:         |
|          |          | [batch_size, height, width, channels] |
|          |          | ex: [10, 1080, 1920, 3]               |
|    N     | n-dim    | You get the idea...                   |
|          | Tensor   |                                       |
---------------------------------------------------------------