CODEWITHSUNDEEP
pythonlogistic-regression
Melad
Melad

Reputation: 3

RuntimeWarning: overflow encountered in exp predictions = 1 / (1 + np.exp(-predictions))

this is the code I'm trying to implement for the dataset file and as I mentioned before the result just gives a 0 and the error :

RuntimeWarning: overflow encountered in exp predictions = 1 / (1 + np.exp(-predictions))

I tried many solutions for other codes related with this prediction but still the same

`import numpy as np
import pandas as pd

dataset = pd.read_csv('data.csv')

dataset = (dataset - dataset.mean()) / dataset.std()

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(dataset.iloc[:, :-1], dataset.iloc[:, -1], test_size=0.25, random_state=42)

def logisticRegression_model(X, y, learning_rate, num_epochs):
    
    weights = np.zeros(X.shape[1])
    
    for epoch in range(num_epochs):
        
        logisticRegression_update_weights(X, y, weights, learning_rate)
    
    return weights

def logisticRegression_update_weights(X, y, weights, learning_rate):
    
    gradient = logisticRegression_calculate_gradient(X, y, weights)
    
    weights += learning_rate * gradient
    
    return weights

def logisticRegression_calculate_gradient(X, y, weights):
    
    #calculating the predictions
    predictions = logisticRegression_predict(X, weights)
    
    #calculating the errors
    error = y - predictions
    
    gradient = np.dot(X.T, error)

    return gradient

def logisticRegression_predict(X, weights):
    
    predictions = np.dot(X, weights)
    
    predictions = 1 / (1 + np.exp(-predictions))
    
    return predictions

def logisticRegression_accuracy(y_true, y_pred):
    
    accuracy = np.sum(y_true == y_pred) / len(y_true)
    
    return accuracy

def logisticRegression_train(X_train, y_train, learning_rate, num_epochs):
    
    weights = logisticRegression_model(X_train, y_train, learning_rate, num_epochs)
    
    return weights

weights = logisticRegression_train(X_train, y_train, 0.1, 1000)

y_pred_train = logisticRegression_predict(X_train, weights)
y_pred_test = logisticRegression_predict(X_test, weights)

y_pred_train = (y_pred_train > 0.5).astype(int)
y_pred_test = (y_pred_test > 0.5).astype(int)

acc_train = logisticRegression_accuracy(y_train, y_pred_train)
acc_test = logisticRegression_accuracy(y_test, y_pred_test)

print('Train accuracy:', acc_train)
print('Test accuracy:', acc_test)`

Upvotes: 0

Views: 978

Answers (1)

C-3PO
C-3PO

Reputation: 1213

This warning occurs because the exponential function exceeds the maximum value accepted for Floating Point (FP) numbers. FP numbers have a limited number of bits to store their exponent in scientific notation, so they can eventually overflow.

This warning is relatively common, and it has no serious consequences (numpy is smart enough to handle the situation, and realize if the number actually corresponds to inf, nan, 0, etc.).

You can even supress the warning message as follows:

import numpy as np
import warnings
warnings.filterwarnings('ignore')
print(1/np.exp(999999999999))

https://www.statology.org/runtimewarning-overflow-encountered-in-exp/#:~:text=This%20warning%20occurs%20when%20you,provides%20the%20warning%20by%20default.

Unfortunately, the issue in the OP code is related to another problem (that is not giving the right result).

PS. If you wrote a code where warnings should not occur at all (because they are related to numerical issues, bugs, etc), you can also transform all numpy warnings into system errors:

numpy.seterr(all='raise')

Now the previous code would crash:

print(1/np.exp(999999999999))
FloatingPointError: overflow encountered in exp

Upvotes: 0

Related Questions