CODEWITHSUNDEEP
scikit-learntext-classification
Hassan Saif
Hassan Saif

Reputation: 1072

Extract most important features (per Class) using mutual_info_classif

I'm using mutual_info_classif to determine the most important words for a binary text-classification task as:

mi_score = mutual_info_classif(X, y)

but the above gives an array of feature scores without reference to the corresponding classes

Is there a way to get the most important features per class using MI?

P.s., I've already tried Chi2 but it gives the same feature rank for both classes

Upvotes: 0

Views: 807

Answers (1)

Roee Anuar
Roee Anuar

Reputation: 3448

Mutual information is a measure of dependence between 2 variables. In your case, between each of the attribute variables and the "Class" variable. The mutual information will give a higher score, when the attribute variable creates a better split of the target variable. This means you only get one score that describes the strength between the attribute and the class. The most important feature is the one that best distinguishes between all of the classes.

If you have a class with multiple labels (not a binary class), you can create a new class variable for each label by using dummy variables. For example, assume your class name is CLASS, and it holds 3 different labels: "Red", "Green" and "Blue". Create 3 new target variables, the first will be called "Is_Red", and it will hold "Yes" if CLASS=="Red" or "No" Otherwise. In this manner you can see which attribute best distinguish between each specific instance of the class. You will have to run the mutual information per new class variable.

Upvotes: 1

Related Questions