Variable sized input doesn't work with dilated convolutional layer

Question

I have a pretty simple model that runs some convolutional layers over text input(embeddings) which have variable sized padded by the batch. I wanted to add some dilated layers but I run into problems.

When running the following code, I can input what ever dimension in the first run, but applying the same conv layers to another input I get errors.

bs = 5
text_len_1 = 772
text_len_2 = 741
embed_size = 300
in_channels = 1

test_in_1 = tf.random.normal((bs, text_len_1, embed_size, in_channels))
test_in_2 = tf.random.normal((bs, text_len_2, embed_size, in_channels))


dilated_convs = [tf.keras.layers.Conv2D(filters=10, kernel_size=(2, embed_size),
                                                    dilation_rate=(dilation, 1),
                                                    padding='valid')
                             for dilation in range(2, 23)]

for conv in dilated_convs:
    res = conv(test_in_1)

for conv in dilated_convs:
    # Fails here, regardless of test_in_1 or 2 is called first
    res = conv(test_in_2) 

At first I thought that I just needed to pad the input properly, but I cant seem to find any pattern I how that should be done, so it might be a simple calcuation needed in order to pad the input properly, but it just seems like there something else going on since im able to call with whatever input as long as its the first input.

EDIT: Added error and stacktrace

2019-06-25 13:50:33.329503: W tensorflow/core/framework/op_kernel.cc:1546] OP_REQUIRES failed at spacetobatch_op.cc:219 : Invalid argument: padded_shape[0]=741 is not divisible by block_shape[0]=2
Traceback (most recent call last):
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/IPython/core/interactiveshell.py", line 3296, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-2-59b677f0b758>", line 1, in <module>
    runfile('/home/name/.PyCharm2019.1/config/scratches/dilated_conv_test.py', wdir='/home/name/.PyCharm2019.1/config/scratches')
  File "/snap/pycharm-professional/136/helpers/pydev/_pydev_bundle/pydev_umd.py", line 197, in runfile
    pydev_imports.execfile(filename, global_vars, local_vars)  # execute the script
  File "/snap/pycharm-professional/136/helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
    exec(compile(contents+"\n", file, 'exec'), glob, loc)
  File "/home/name/.PyCharm2019.1/config/scratches/dilated_conv_test.py", line 22, in <module>
    res = conv(test_in_2)
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/tensorflow/python/keras/engine/base_layer.py", line 712, in __call__
    outputs = self.call(inputs, *args, **kwargs)
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/tensorflow/python/keras/layers/convolutional.py", line 196, in call
    outputs = self._convolution_op(inputs, self.kernel)
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 1078, in __call__
    return self.conv_op(inp, filter)
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 634, in __call__
    return self.call(inp, filter)
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/tensorflow/python/ops/nn_ops.py", line 617, in _with_space_to_batch_call
    input=inp, block_shape=dilation_rate, paddings=paddings)
  File "/home/name/anaconda3/envs/3.6/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py", line 9246, in space_to_batch_nd
    _six.raise_from(_core._status_to_exception(e.code, message), None)
  File "<string>", line 3, in raise_from
tensorflow.python.framework.errors_impl.InvalidArgumentError: padded_shape[0]=741 is not divisible by block_shape[0]=2 [Op:SpaceToBatchND]

Answer 1

I ran your code and the error persists in tensorflow version 1.x but vanishes when upgraded to tensorflow version 2.x. Can you please upgrade to 2.x version and try.

Below are the run details.

Run 1 - Tensorflow Version 1.x

%tensorflow_version 1.x
import tensorflow as tf

bs = 5
text_len_1 = 772
text_len_2 = 741
embed_size = 300
in_channels = 1

test_in_1 = tf.random.normal((bs, text_len_1, embed_size, in_channels))
test_in_2 = tf.random.normal((bs, text_len_2, embed_size, in_channels))

dilated_convs = [tf.keras.layers.Conv2D(filters=10, kernel_size=(2, embed_size),
                                                    dilation_rate=(dilation, 1),
                                                    padding='same')
                             for dilation in range(2, 23)]

print(dilated_convs)
for conv in dilated_convs:
    res = conv(test_in_1)

for conv in dilated_convs:
    # Fails here, regardless of test_in_1 or 2 is called first
    res = conv(test_in_2) 

print("I ran Succesfully")

Output -

TensorFlow 1.x selected.
[<tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe3db797160>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa30e48>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa542b0>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa546d8>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa54b00>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa54f28>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa58390>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa587b8>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa58be0>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa5f048>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa5f470>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa5f898>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa5fcc0>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa62128>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa62550>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa62978>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39fa62da0>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39f9e8208>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39f9e8630>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39f9e8a58>, <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x7fe39f9e8e80>]
WARNING:tensorflow:From /tensorflow-1.15.2/python3.6/tensorflow_core/python/ops/resource_variable_ops.py:1630: calling BaseResourceVariable.__init__ (from tensorflow.python.ops.resource_variable_ops) with constraint is deprecated and will be removed in a future version.
Instructions for updating:
If using Keras pass *_constraint arguments to layers.
---------------------------------------------------------------------------
InvalidArgumentError                      Traceback (most recent call last)
/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1606   try:
-> 1607     c_op = c_api.TF_FinishOperation(op_desc)
   1608   except errors.InvalidArgumentError as e:

InvalidArgumentError: Dimension size must be evenly divisible by 2 but is 743 for 'conv2d_21/SpaceToBatchND' (op: 'SpaceToBatchND') with input shapes: [5,741,300,1], [2], [2,2] and with computed input tensors: input[1] = <2 1>, input[2] = <[1 1][149 150]>.

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
12 frames
/tensorflow-1.15.2/python3.6/tensorflow_core/python/framework/ops.py in _create_c_op(graph, node_def, inputs, control_inputs)
   1608   except errors.InvalidArgumentError as e:
   1609     # Convert to ValueError for backwards compatibility.
-> 1610     raise ValueError(str(e))
   1611 
   1612   return c_op

ValueError: Dimension size must be evenly divisible by 2 but is 743 for 'conv2d_21/SpaceToBatchND' (op: 'SpaceToBatchND') with input shapes: [5,741,300,1], [2], [2,2] and with computed input tensors: input[1] = <2 1>, input[2] = <[1 1][149 150]>.

Run 2 - Tensorflow Version 2.x

%tensorflow_version 2.x
import tensorflow as tf

bs = 5
text_len_1 = 772
text_len_2 = 741
embed_size = 300
in_channels = 1

test_in_1 = tf.random.normal((bs, text_len_1, embed_size, in_channels))
test_in_2 = tf.random.normal((bs, text_len_2, embed_size, in_channels))

dilated_convs = [tf.keras.layers.Conv2D(filters=10, kernel_size=(2, embed_size),
                                                    dilation_rate=(dilation, 1),
                                                    padding='same')
                             for dilation in range(2, 23)]

for conv in dilated_convs:
    res = conv(test_in_1)

for conv in dilated_convs:
    # Fails here, regardless of test_in_1 or 2 is called first
    res = conv(test_in_2) 

print("I ran Succesfully")

Output -

I ran Succesfully