Keras LSTM has different output for single input

Question

I came to a weird problem when using Keras LSTM model. I build a single layer LSTM and try to play with it. I found the output of the model is different between one input and multiple inputs, as shown in the following code.

def lstmTest(training, latent_dim=10):
        _, time_dim, input_dim = training.shape

        # Define an input sequence and process it.
        encoder_inputs = Input(shape=(time_dim, input_dim), name='input')
        encoder = LSTM(latent_dim, return_state=False, name='lstm')
        encoder_outputs = encoder(encoder_inputs)

        model = Model(encoder_inputs, encoder_outputs)

        return model

def trainingTest(model, training, nb_epoch=10, batch_size=300):
        model.compile(optimizer='adam', loss='mse', metrics=['acc'])
        history = model.fit(training, training[:, -1, :10],
                            epochs=nb_epoch,
                            batch_size=batch_size,
                            shuffle=True,
                            verbose=1,
                            ).history
        return history

myVector = [[[i]*20]*8 for i in range(100)]
myVector = np.array(myVector)

lstmTest = lstmTest(myVector)
history = trainingTest(lstmTest, myVector)

vector = myVector[:2]
res1 = lstmTest.predict(vector)

vector = myVector[:1]
res2 = lstmTest.predict(vector)

res2[0] - res1[0]

I got the following result

array([0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 5.8207661e-11,
       0.0000000e+00, 2.3283064e-10, 0.0000000e+00, 0.0000000e+00,
       0.0000000e+00, 0.0000000e+00], dtype=float32)

But If I change res2 to the same as res1 I got the expected result

array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.], dtype=float32)

Anyone came to this problem before?

Answer 1

There is no difference, it´s just the limit of IEEE double values:

print(0.003f - 0.001f - 0.002f)
-2.3283064E-10

If you are using the gpu you can take a look at this