CODEWITHSUNDEEP
machine-learningregressionnormalization
Duc Nguyen
Duc Nguyen

Reputation: 761

Pre-processing data: Normalizing data labels in regression?

Recently I was told that the labels of regression data should also be normalized for better result but I am pretty doubtful of that. I have never tried normalizing labels in both regression and classification that's why I don't know if that state is true or not. Can you please give me a clear explanation (mathematically or in experience) about this problem?

Thank you so much. Any help would be appreciated.

Upvotes: 5

Views: 18054

Answers (4)

Ali karimi
Ali karimi

Reputation: 500

for a regression problem with algorithms including decision tree or logistic regression and linear regression I tested in two modes: 1- with label scaling using MinMaxScaler 2- without label scaling the result that i got was : r2 score is the same in 2 mode mse and mae scales

for diabetes dataset using linear regression the result before and after is

without scaling:

Mean Squared Error: 3424.3166
Mean Absolute Error: 46.1742

R2_score : 0.33

after scaling labels:

Mean Squared Error: 0.0332
Mean Absolute Error: 0.1438
R2_score : 0.33

also below link can be useful which says scaling can be helpful in fast convergence enter scale or not scale labels in deep leaning?

Upvotes: 0

rodrigo-silveira
rodrigo-silveira

Reputation: 13078

It may be that what you mean is that you should scale your labels. The reason is so convergence is faster, and you don't get numeric instability.

For example, if your labels are in the range (1000, 1000000) and the weights are initialized close to zero, a mse loss would be so large, you'd likely get NaN errors.

See https://datascience.stackexchange.com/q/22776/38707 for a similar discussion.

Upvotes: 3

slouchy
slouchy

Reputation: 336

When you say "normalize" labels, it is not clear what you mean (i.e. whether you mean this in a statistical sense or something else). Can you please provide an example?

On Making labels uniform in data analysis If you are trying to neaten labels for use with the text() function, you could try the abbreviate() function to shorten them, or the format() function to align them better.

The pretty() function works well for rounding labels on plot axes. For instance, the base function hist() for drawing histograms calls on Sturges or other algorithms and then uses pretty() to choose nice bin sizes.

The scale() function will standardize values by subtracting their mean and dividing by the standard deviation, which in some circles is referred to as normalization.

On the reasons for scaling in regression (in response to comment by questor). Suppose you regress Y on covariates X1, X2, ... The reasons for scaling covariates Xk depend on the context. It can enable comparison of the coefficients (effect sizes) of each covariate. It can help ensure numerical accuracy (these days not usually an issue unless covariates on hugely different scales and/or data is big). For a readable intro see Psychosomatic medicine editors' guide. For a mathematically intense discussion see Sylvain Sardy's guide.

In particular, in Bayesian regression, rescaling is advisable to ensure convergence of MCMC estimation; e.g. see this discussion.

Upvotes: 2

Hatim Khouzaimi
Hatim Khouzaimi

Reputation: 511

You mean features not labels.

It is not necessary to normalize your features for regression or classification, even though in some cases, it is a trick that can help converging faster. You might want to check this post.

To my experience, when using a simple model like a linear regression with only a few variables, keeping the features as they are (without normalization) is preferable since the model is more interpretable.

Upvotes: -1

Related Questions