CODEWITHSUNDEEP
pythonkerasdeep-learningtime-serieslstm
Kumar
Kumar

Reputation: 53

Issue with concatenating Keras layers

I am trying to build an LSTM model to predict temperature for a given day using say past 7 days of temperature, rainfall etc of a Zipcode or PinCode. I understand that the training dataset needs to be shaped as (observations, timesteps, features). I guess the features in my case would be the temperature, rainfall and timesteps would be 7. So If I had 2 features and 7 timesteps, there would be a total of 14 variables for one observation. I also have features like State, Zone (say North, West, East, Central etc) which are common to all the 7 timesteps. Since each observation is a zipcode, for all 7 days (7 timesteps) of a zipcode, Geographical State & Zone to which the zip code belongs, would be the same. Since we can’t input features (common to all timesteps) to LSTM, I have used the Keras functional API to feed those temporal features to LSTM and take LSTM’s output and concatenate that with non-temporal features (State, Zone etc) and then have a dense layer at the end. Below is my code

Input_LSTM = Input(shape=(7, 2), batch_size=32)  # 7 timesteps and 2 features
x = LSTM(5,  stateful=True)(Input_LSTM)
x = Dropout(0.2)(x)

Input_MLP = Input(shape=(261,),batch_size=32)  
x = concatenate([x,Input_MLP], axis=1)  # Join non-temporal input with LSTM o/p to be fed to a Dense    
# layer
Output = Dense(1)(x)

model = Model([Input_LSTM,Input_MLP],Output)

Since I have declared LSTM to be stateful, I have given batch_size for both temporal and non-temporal inputs. When I try to concatenate using the line “x = concatenate([x,Input_MLP], axis=1)” , I am getting error

  File "<__array_function__ internals>", line 6, in concatenate

ValueError: zero-dimensional arrays cannot be concatenated”

The shapes of both elements of concatenation are as below

x.shape
Out[186]: TensorShape([Dimension(32), Dimension(5)])      #  Here 5 is the number of neurons in LSTM, 32 is the batch size

Input_MLP.shape
Out[187]: TensorShape([Dimension(32), Dimension(261)])   # Here 261 is the no. of non-temporal features and 32 is the batch size

Found from net that zero-dimensional arrays are scalars. But from their shapes above, they don’t seem to be scalars. I also tried a Dense layer with Input_MLP as input and tried to concatenate this Dense layer’s output with x but in vain.

I could not find an answer in the net. What am I doing wrong? Any help would be greatly appreciated.

Upvotes: 0

Views: 153

Answers (1)

Kumar
Kumar

Reputation: 53

I found where the issue is. The function to concatenate keras layers must be Concatenate with Capital C. Since I was using the one with lowercase 'c', it was numpy function and hence it was giving that error. A small typo costed me lot of time

Upvotes: 1

Related Questions