CODEWITHSUNDEEP
pythontensorflowkerasbackend
Sandeep Pandey
Sandeep Pandey

Reputation: 384

Keras repeat_elements with unknown batch_size

I have a function where i need to do Keras batch_dot of a tensor of size (?,61,80) with 2D tensor of size (40,61). Dimension ? is for the batch size in custom layer. While using Keras repeat_elements, we need to specify batch size to make it a tensor of (batch_size, 40,61). However, repeat_elements doesn't work with ? batch size.

The code is 

M1 = K.expand_dims(M,axis=0)
BatchM = K.repeat_elements(x=M1,rep=batch_size,axis=0)
out1 = K.batch_dot(BatchM,Ash1,axes=[2,1])

Here M is 2D tensor of size (40,61). BatchM should give (batch_size,40,61) and Ash1 is of size (?,61,80).

Edit 1:

A= Input(shape=(61,80))
M= K.variable(np.random.rand(40,61))
n=1

import tensorflow as tf
M1 = K.expand_dims(M,axis=0)
BatchM = K.repeat_elements(x=M1,rep=tf.shape(A)[0],axis=0)
out1 = K.batch_dot(BatchM,Ash1,axes=[2,1])

This return error shows:

Traceback (most recent call last)

 File "<ipython-input-7-edc5ef31181b>", line 3, in <module>
    BatchM = K.repeat_elements(x=M1,rep=tf.shape(A)[0],axis=0)

  File "/home/hanumant/.conda/envs/kerasenv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2092, in repeat_elements
    x_rep = [s for s in splits for _ in range(rep)]

  File "/home/hanumant/.conda/envs/kerasenv/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py", line 2092, in <listcomp>
    x_rep = [s for s in splits for _ in range(rep)]

TypeError: 'Tensor' object cannot be interpreted as an integer

Upvotes: 2

Views: 1798

Answers (1)

giser_yugang
giser_yugang

Reputation: 6166

In fact, you need not to repeat_elements with unknown batch_size. You can use K.dot() and K.permute_dimensions directly for the same purpose.

def customer_dot(a,b):
    a = K.permute_dimensions(a, (0, 2, 1))  # x = (?,80,61)
    b = K.permute_dimensions(b, (1, 0))  # kernel = (61,40)
    ab_dot = K.permute_dimensions(K.dot(a, b), (0, 2, 1)) # ab_dot = (?,40,80)
    return ab_dot

A = Input(shape=(61,80))
M = K.variable(np.random.rand(40,61))

result = customer_dot(A,M)
print(result.shape)

# print
(?, 40, 80)

And you can use the following examples to see that the result is the same as that of your code operation.

# print
A = K.constant(np.random.rand(3,2,4))
M = K.constant(np.random.rand(5,2))

M1 = K.expand_dims(M,axis=0)
BatchM = K.repeat_elements(x=M1,rep=K.int_shape(A)[0],axis=0)
out1 = K.batch_dot(BatchM,A,axes=[2,1])
print(K.eval(out1))
result = customer_dot(A,M)
print(K.eval(result))

[[[0.07588554 0.19896106 0.4122516  0.16694324]
  [0.02837059 0.07994501 0.15250334 0.05631477]
  [0.02922964 0.03180532 0.17185953 0.11346529]
  [0.24399586 0.64474815 1.3240533  0.53126353]
  [0.06582426 0.0952256  0.38014278 0.22963922]]

 [[0.05856805 0.31629622 0.37190455 0.15167782]
  [0.02006819 0.12145159 0.1384899  0.0497717 ]
  [0.03729554 0.09602766 0.14768752 0.11432388]
  [0.18666261 1.0198846  1.1952925  0.481425  ]
  [0.07623056 0.2298356  0.33025196 0.22802524]]

 [[0.29545793 0.27023914 0.14775626 0.22487558]
  [0.10839225 0.10083499 0.05140937 0.07595014]
  [0.13047284 0.10567644 0.08779343 0.15208915]
  [0.9481214  0.868726   0.47162086 0.7157058 ]
  [0.28504598 0.23714545 0.18145116 0.30803293]]]
[[[0.07588554 0.19896106 0.4122516  0.16694324]
  [0.02837059 0.07994501 0.15250334 0.05631477]
  [0.02922964 0.03180532 0.17185953 0.11346529]
  [0.24399586 0.64474815 1.3240533  0.53126353]
  [0.06582426 0.0952256  0.38014278 0.22963922]]

 [[0.05856805 0.31629622 0.37190455 0.15167782]
  [0.02006819 0.12145159 0.1384899  0.0497717 ]
  [0.03729554 0.09602766 0.14768752 0.11432388]
  [0.18666261 1.0198846  1.1952925  0.481425  ]
  [0.07623056 0.2298356  0.33025196 0.22802524]]

 [[0.29545793 0.27023914 0.14775626 0.22487558]
  [0.10839225 0.10083499 0.05140937 0.07595014]
  [0.13047284 0.10567644 0.08779343 0.15208915]
  [0.9481214  0.868726   0.47162086 0.7157058 ]
  [0.28504598 0.23714545 0.18145116 0.30803293]]]

Upvotes: 2

Related Questions