Different results when training sklearn logistic regression iteratively (epoch by epoch)

Question

I am training sklearn logistic regression iteratively (the motivation being that I can return the mean-squared error after each epoch this way, which unfortunately cannot be otherwise retrieved). However, I get slightly different metrics (e.g. F1 score) when doing it this way than when just doing one call of model.fit(). Does anyone know why this is?

Typical way of training:

    model = LogisticRegression(
            random_state=42,
            C=.001,
            penalty='l1',
            max_iter=500,
            solver='saga'))

    model.fit(X_train, y_train)

    # Predict Class labels
    pred_labels_val = model.predict(X_val)
    pred_labels_test = model.predict(X_test)

    # F1 Score
    f1_val = metrics.f1_score(y_val, pred_labels_val)
    f1_test = metrics.f1_score(y_test, pred_labels_test)

Alternatively, I am training iteratively to get the MSE after each epoch. To see if the answers are the same, I set the number of epochs to the number at which the previous model converged (otherwise it would be forced to train for the full number which could cause different results). Everything else remains the same.

    epochs = model.named_steps['logisticregression'].n_iter_[0]

    model = LogisticRegression(
            random_state=42,
            C=.001,
            penalty='l1',
            max_iter=1,  # this forces training to run for only one epoch
            solver='saga'))

    for epoch in range(epochs): 
        model.fit(X_train, y_train)
        
    # Predict Class labels
    pred_labels_val = model.predict(X_val)
    pred_labels_test = model.predict(X_test)

    # F1 Score
    f1_val = metrics.f1_score(y_val, pred_labels_val)
    f1_test = metrics.f1_score(y_test, pred_labels_test)

However, the resulting metrics (e.g. F1) are different.

Answer 1

first, you use "model" to fit, but use "model_log" to predict, which is confusing and maybe a mistake. And so I am not sure your "epochs" is correct.

Second, fitting the model many times seems strange, it looks like repeatedly training the model in the same way. A "fit" may have many epochs, so this for loop may be meaningless.