STN: spatial trasformer network: Rotate and translate image

Question

CNN is:

• input: 2 image 128x128: input1 and input2
• output: CNN return a Trasformation Matrix:

---

outLayer = Layers.Dense(3 * 2, activation=activations.linear,kernel_initializer="zeos",bias_initializer=output_bias)(d2)

---

After I:

• append STN (https://github.com/kevinzakka/spatial-transformer-network) for translate and rotate input1. The complete network is CNN with STN. Return an image (128x128) with 32 batch
• My loss is MSE between input2 and output (CNN): target error registration. MSE between 2 image (input2 and output)

The network must find the transformation matrix because input2 = output Can I use STN only for translate and rotate input1? Because I have problem with val_loss (use tensorflow):

---

Epoch 1/90 30/30 [==============================] - 81s 3s/step - loss: 16.0190 - val_loss: 24.3248

Epoch 2/90 30/30 [==============================] - 79s 3s/step - loss: 13.9868 - val_loss: 21.4465

Epoch 3/90 30/30 [==============================] - 73s 2s/step - loss: 13.2970 - val_loss: 21.3151

Epoch 4/90 30/30 [==============================] - 69s 2s/step - loss: 12.9244 - val_loss: 21.6154

Epoch 5/90 30/30 [==============================] - 67s 2s/step - loss: 12.6868 - val_loss: 20.0113

Epoch 6/90 30/30 [==============================] - 66s 2s/step - loss: 12.4998 - val_loss: 20.8911

Epoch 7/90 30/30 [==============================] - 69s 2s/step - loss: 12.3066 - val_loss: 21.4276

Epoch 8/90 30/30 [==============================] - 67s 2s/step - loss: 12.1034 - val_loss: 21.3593

Epoch 9/90 30/30 [==============================] - 69s 2s/step - loss: 11.7645 - val_loss: 20.8941

Epoch 10/90 30/30 [==============================] - 67s 2s/step - loss: 11.6544 - val_loss: 20.4768

Epoch 11/90 30/30 [==============================] - 70s 2s/step - loss: 11.5483 - val_loss: 21.7420

Epoch 12/90 30/30 [==============================] - 68s 2s/step - loss: 11.5680 - val_loss: 19.4531

---