CODEWITHSUNDEEP
scikit-learnregressionscalingsklearn-pandas
Yann
Yann

Reputation: 191

Questions about standardizing and scaling

I am trying to generate a model that uses several physico-chemical properties of a molecule (incl. number of atoms, number of rings, volume, etc.) to predict a numeric value Y. I would like to use PLS Regression, and I understand that standardization is very important here. I am programming in Python, using scikit-learn. The type and range for the features varies. Some are int64 while other are float. Some features generally have small (positive or negative) values, while other have very large value. I have tried using various scalers (e.g. standard scaler, normalize, minmax scaler, etc.). Yet, the R2/Q2 are still low. I have a few questions:

  1. Is it possible that by scaling, some of the very important features lose their significance, and thus contribute less to explaining the variance of the response variable?
  2. If yes, if I identify some important features (by expert knowledge), is it OK to scale other features but those? Or scale the important features only?
  3. Some of the features, although not always correlated, have values that are in a similar range (e.g. 100-400), compared to others (e.g. -1 to 10). Is it possible to scale only a specific group of features that are within the same range?

Upvotes: 1

Views: 691

Answers (2)

Ankish Bansal
Ankish Bansal

Reputation: 1902

The whole idea of scaling is to make models more robust to analysis on features space. For example, if you have 2 features as 5 Kg and 5000 gm, we know both are same, but for some algorithm, which are sensitive to metric space such as KNN, PCA etc, they will be more weighted towards second features, so scaling must be done for these algos.

Now coming to your question,

  1. Scaling doesn't effect the significance of features. As i explained above, it helps in better analysis of data.
  2. No, you should not do, reason explained above.
  3. If you want to include domain knowledge in your model, you can use it as prior information. In short, for linear model, this is same as regularization. It has very good features. if you think, you have many useless-features, you can use L1 regularization, which creates sparse effect on features space, which is nothing but assign 0 weight to useless features. Here is the link for more-info.

One more point, some method such as tree based model doesn't need scaling, In last, it mostly depend on the model, you choose.

Upvotes: 2

user2974951
user2974951

Reputation: 10375

  1. Lose significance? Yes. Contribute less? No.
  2. No, it's not OK. It's either all or nothing.
  3. No. The idea of scaling is not to decrease / increase significance / effect of a variable. It's to transform all variables to a common scale that can be interpreted.

Upvotes: 1

Related Questions