Input Dimensions Tensorflow v1.8 ConvLSTMCell

Question

ConvLSTMCell Official Docs

GitHub _conv where the error occurs

Issue

I'm experimenting with the ConvLSTMCell in tensorflow r1.8. The error I'm continuing to generate occurs in the __call__ method of ConvLSTMCell. The _conv method is invoked and the error is raised.

ValueError: Conv Linear Expects 3D, 4D, 5D

The error is raised from the unstacked inputs. unstacked (in this example) has dimensions of [BATCH_SIZE, N_INPUTS] = [2,5]. I am using tf.unstack to generate the required sequence that the ConvLSTMCell requires.

Why use tf.unstack?

If the input array is not unstacked, the TypeError below is raised.

TypeError: inputs must be a sequence

Question

What am I missing on the formatting? I've read through related issues but have not found anything that has guided me into a working implementation.

• Are the placeholder dimensions correct?

• Should I be unstacking or is there a better way?

• Am I providing the proper input dimension into the ConvLSTMCell?

Code

# Parameters
TIME_STEPS = 28
N_INPUT = 5
N_HIDDEN = 128
LEARNING_RATE = 0.001
NUM_UNITS = 28
CHANNEL = 1
tf.reset_default_graph()

# Input placeholders
x = tf.placeholder(tf.float32, [BATCH_SIZE, TIME_STEPS, N_INPUT])
y = tf.placeholder(tf.float32, [None, 1])

# Format input as a sequence for LSTM Input
unstacked = tf.unstack(x, TIME_STEPS, 1) # shape=(timesteps, batch, inputs)

# Convolutional LSTM Layer
lstm_layer = tf.contrib.rnn.ConvLSTMCell(
        conv_ndims=1,
        input_shape=[BATCH_SIZE, N_INPUT],
        output_channels=5,
        kernel_shape=[7,5]
    )

# Error is generated when the lstm_layer is invoked 
outputs, _ = tf.contrib.rnn.static_rnn(
    lstm_layer, 
    unstacked,
    dtype=tf.float32) 

Error Message

    ---------------------------------------------------------------------------
    ValueError                                Traceback (most recent call last)
    <ipython-input-83-3568a097e4ea> in <module>()
         10     lstm_layer,
         11     unstacked,
    ---> 12     dtype=tf.float32) 

    ~/miniconda3/envs/MultivariateTimeSeries/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py in static_rnn(cell, inputs, initial_state, dtype, sequence_length, scope)
       1322             state_size=cell.state_size)
       1323       else:
    -> 1324         (output, state) = call_cell()
       1325 
       1326       outputs.append(output)

    ~/miniconda3/envs/MultivariateTimeSeries/lib/python3.6/site-packages/tensorflow/python/ops/rnn.py in <lambda>()
       1309         varscope.reuse_variables()
       1310       # pylint: disable=cell-var-from-loop
    -> 1311       call_cell = lambda: cell(input_, state)
       1312       # pylint: enable=cell-var-from-loop
       1313       if sequence_length is not None:

    ~/miniconda3/envs/MultivariateTimeSeries/lib/python3.6/site-packages/tensorflow/python/ops/rnn_cell_impl.py in __call__(self, inputs, state, scope)
        230         setattr(self, scope_attrname, scope)
        231       with scope:
    --> 232         return super(RNNCell, self).__call__(inputs, state)
        233 
        234   def _rnn_get_variable(self, getter, *args, **kwargs):

    ~/miniconda3/envs/MultivariateTimeSeries/lib/python3.6/site-packages/tensorflow/python/layers/base.py in __call__(self, inputs, *args, **kwargs)
        715 
        716         if not in_deferred_mode:
    --> 717           outputs = self.call(inputs, *args, **kwargs)
        718           if outputs is None:
        719             raise ValueError('A layer\'s `call` method should return a Tensor '

    ~/miniconda3/envs/MultivariateTimeSeries/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/rnn_cell.py in call(self, inputs, state, scope)
       2110     cell, hidden = state
       2111     new_hidden = _conv([inputs, hidden], self._kernel_shape,
    -> 2112                        4 * self._output_channels, self._use_bias)
       2113     gates = array_ops.split(
       2114         value=new_hidden, num_or_size_splits=4, axis=self._conv_ndims + 1)

    ~/miniconda3/envs/MultivariateTimeSeries/lib/python3.6/site-packages/tensorflow/contrib/rnn/python/ops/rnn_cell.py in _conv(args, filter_size, num_features, bias, bias_start)
       2184     if len(shape) not in [3, 4, 5]:
       2185       raise ValueError("Conv Linear expects 3D, 4D "
    -> 2186                        "or 5D arguments: %s" % str(shapes))
       2187     if len(shape) != len(shapes[0]):
       2188       raise ValueError("Conv Linear expects all args "

    ValueError: Conv Linear expects 3D, 4D or 5D arguments: [[2, 5], [2, 2, 5]]

Answer 1

Here's an example with a couple tweaks, which at least passes static shape checking:

import tensorflow as tf

# Parameters
TIME_STEPS = 28
N_INPUT = 5
N_HIDDEN = 128
LEARNING_RATE = 0.001
NUM_UNITS = 28
CHANNEL = 1
BATCH_SIZE = 16

# Input placeholders
x = tf.placeholder(tf.float32, [BATCH_SIZE, TIME_STEPS, N_INPUT])
y = tf.placeholder(tf.float32, [None, 1])

# Format input as a sequence for LSTM Input
unstacked = tf.unstack(x[..., None], TIME_STEPS, 1) # shape=(timesteps, batch, inputs)

# Convolutional LSTM Layer
lstm_layer = tf.contrib.rnn.ConvLSTMCell(
        conv_ndims=1,
        input_shape=[N_INPUT, 1],
        output_channels=5,
        kernel_shape=[7]
    )

# Error is generated when the lstm_layer is invoked
outputs, _ = tf.contrib.rnn.static_rnn(
    lstm_layer,
    unstacked,
    dtype=tf.float32)

Notes:

• input_shape does not include the batch dimension (see docstring)
• The input needs a channels dimension. Fine for it to be one in the input (that's what I've done).
• Not sure what more than one dimension on kernel_shape would mean for a 1-D convolution.