CODEWITHSUNDEEP
rmachine-learningdata-miningdata-modeling
tony
tony

Reputation: 1257

What does the information gain measure show?

I am busy using the fSelector package in r to measure information gain using the information.gain function.

I am a unsure of what the output is giving me as there are minuses, I understand the output on the iris dataset.

weights <- information.gain(cross_over ~ age + max_stake_scale + current_loyalty_status + lifespan + early_hours + morning + afternoon + evening, df

The output is: 

attr_importance
age                           0.000000e+00
max_stake_scale               0.000000e+00
current_loyalty_status        2.968854e-04
lifespan                      1.523364e-02
early_hours                   0.000000e+00
morning                       0.000000e+00
afternoon                     0.000000e+00
evening                       0.000000e+00

The response variable is binary, cross or not, yes or no. I am unsure of what the important features are please.

Thanks

Upvotes: 0

Views: 454

Answers (1)

Lars Kotthoff
Lars Kotthoff

Reputation: 109242

The information gain tells you how much information you gain by getting to know the value of a specific feature. In the case of a binary classification problem, you need 1 Bit of information to make the classification (as there are two possible outcomes); i.e. the maximum information gain for a specific feature is 1.

In your case, the information gain for almost all of the features is 0, meaning that knowing their value doesn't help at all with deciding which class the datum has. The non-zero values denote the most important (albeit still quite uninformative) features -- lifespan and current_loyalty_status. The minuses are simply part of the exponential notation, i.e. 2.968854e-04 means 2.968854 * 10^-4 or 0.0002968854.

Upvotes: 3

Related Questions