CODEWITHSUNDEEP
pythontensorflowkerasneural-network
Djam75
Djam75

Reputation: 31

Neural Network optimization using epoch and batch

I am trying to optimize a given neural network (ex Perceptron Multilayer, with 2 hidden layers), by finding the number of epoch and batch that give the highest accuracy.

   for epoch from 10 to 200 (in steps of 10):
         for batch from 40 to 200 (in steps of 20):
                modele.fit (X_train, Y_train, epochs = epoch, batch_size = batch)
                I save batch, epoch, Accuracy;

Afterwards I kept the smallest epoch with the smallest corresponding batch which has the highest recognition ex best_params: epoch = 10, batch = 150 => Accuracy = 94%

My problem is that when I re-run my model with the best_params, it doesn't give me the same results (loss, accuracy), even sometimes very low accuracy (eg 10%).

i try to fix seed, but no best result

Regards

Djam75

Upvotes: 1

Views: 132

Answers (4)

Djam75
Djam75

Reputation: 31

Thanks for the update.

I resolve my probelm by saving a model and load it after.

thaks for idea (batch normalization ) and extra idea : not user any for ;-)

regards

Upvotes: 1

nipun
nipun

Reputation: 692

This might be happening due to random parameter initialization. Because if you are building an end-to-end model without transfer learn the weights, every time you training architecture get random values for its parameters.

In this case, a good practice is to use batch normalization layers after some layers according to your architecture.
tensoflow-implementation

pytorch-implmentation

extra idea: Do not use any 'for', 'while' loops in the model implementation. you can follow templates in TensorFlow or PyTorch. OR, if you build a complete model from scratch, vectorize operations by using NumPy like metrics operation library.

Upvotes: 1

Djam75
Djam75

Reputation: 31

df=pd.DataFrame(columns=['Nb_Batch','Nb_Epoch','Accuracy'])
i=0
lst_loss=[]
lst_accuracy=[]
lst_epoch=list(np.arange(10,200,10))
lst_batch=list(np.arange(100,400,20))
for epoch in lst_epoch:
    print ('---------------- Epoch ' + str(epoch)+ '------------------')
    for batch in lst_batch:
        modelSimple.fit(X_train, Y_train, nb_epoch = epoch, batch_size = batch, verbose = 0)
        score = modelSimple.evaluate(X_test, Y_test)
        df.loc[i,"Nb_Batch"]=batch
        df.loc[i,"Nb_Epoch"]=epoch
        df.loc[i,"Accuracy"]=score[1]*100
        i=i+1

Upvotes: 1

Alex Ferguson
Alex Ferguson

Reputation: 107

I think you might not be updating the weight matrix after completing the training for certain batch sizes and epochs.

Please include the code as well in order to see the problem

Upvotes: 0

Related Questions