Improve accuracy on convolutional neural network

Question

I'm trying to train a convolutional neural network to classify sentences. I take the code from here

The code works fine in one of my datasets. However, on another dataset, its performance is very poor. Two datasets are comparable in terms of volumes and sentence length. For the dataset with the poor performance, I notice the loss function decreases after steps. Here the blue is for training set and the read is for test set.

And here is the accuracy: As you can see, the loss values on the test set decrease, but the accuracy values are almost the same. Can you suggest me how to fix it? Thank you very much.

Answer 1

Try using an exponential decaying learning rate--starts big at first (say 0.1) and then slows down over iterations.

Answer 2

Can you try with a smaller learning rate?