How to plot accuracy and loss curves for train and test data in MLPClassifier using sklean?

Question

I am using this very simple code for training MLPClassifier.

x_train, x_test, y_train, y_test = load_data(test_size=0.25)
model = MLPClassifier(alpha=0.01,
                      batch_size=128,
                      epsilon=1e-08,
                      hidden_layer_sizes=(300,),
                      learning_rate='adaptive',
                      max_iter=500,
                      early_stopping=True)
model.fit(x_train, y_train)
y_pred = model.predict(x_test)
accuracy = accuracy_score(y_true=y_test, y_pred=y_pred)

It is perfectly giving accuracy. now my questions are:

• How to get the maximum accuracy for the model, as it is changed every time I run the code?
• Due to the random selection of train test data, is there any way to find the maximum possible accuracy?

secondly, I want to plot the curves for accuracy and loss for train and Val data. I come to know about

plt.plot(model.loss_curve_)
plt.plot(model.validation_scores_)

But don't know how to use them and tried this but why has the val_loss been low since starting:

I have tried the following code from this community only

scores_train = []
scores_test = []

# EPOCH
epoch = 0
while epoch < n_epoch:
    print('epoch: ', epoch)
    # SHUFFLING
    random_perm = np.random.permutation(x_train.shape[0])
    mini_batch_index = 0
    while True:
        # MINI-BATCH
        indices = random_perm[mini_batch_index:mini_batch_index + 128]
        model.partial_fit(x_train[indices], y_train[indices], classes=7)
        mini_batch_index += 128

        if mini_batch_index >= x_train.shape[0]:
            break

    # SCORE TRAIN
    scores_train.append(model.score(x_train, y_train))

    # SCORE TEST
    scores_test.append(model.score(x_test, y_test))

    epoch += 1

""" Plot """

plt.plot(scores_train, color='green', alpha=0.8, label='Train')
plt.plot(scores_test, color='magenta', alpha=0.8, label='Test')
plt.title("Accuracy over epochs", fontsize=14)
plt.xlabel('Epochs')
plt.legend(loc='upper left')
plt.show()

But its throwing an error at line :

model.partial_fit(x_train[indices], y_train[indices], classes=7)

returns:

Error: only integer scalar arrays can be converted to a scalar index

What am I doing wrong please some one guide.

Answer 1

Got the result by just putting

MLPClassifier(early_stopping=False, warm_start=True)

in MLPClassifier(). Don't know much about it, but solved the purpose.